TIL: backtesting
basically you create an algorithm or strategy (generally for trading) and then use that on historical data - what’s already happened - to test the strategy.
could be used for decision making frameworks as well. which framework would led you to have done what during a major decision in the past?
this is obviously better for those datasets that are high quality - easily quantifiable and high-frequency. so naturally stock markets are the first.
demis hassabis said something similar. that we could define agi as a system - in whose training data you feed it only the knowledge humans had 3-4 years prior to when einstein did his groundbreaking work, and it could independently come up with the theory of relativity.