How do I know that my strategy works?
Here are some key points that you should consider when evaluating your trading strategy:
- Did it run only once or it did run many times?
- Is the average of your target like yield and yield stdev converges to a certain number by increasing number of iterations?
- Did you shuffle your stocks at each iteration?
- Is the simulation reproducible e.g. are you using random seeds?
- How did you evaluate the results? Did you do some feature importance analysis on that based on an ML model?
- How many trades were made during the period? Just some at the beginning of the period because of a certain setting or at least 2-3 per week?
All in all, the goal is to have a strategy that performs stable with the prediction signal in the most heterogenous variable environment (setting-configuration)
Evaluation principles
- We evaluate not only experiment but an experiment aggregate that is a result of a simulation with many ~ 30 iterations
- An experiment is valid if it had a reliable and large symbol pool ~ 50 stocks available per iteration
- An experiment is reliable if the overall yield stdev is ~20% of the mean yield
- Experiments are always compared to experiments without using prediction to decide purchase and sp500 and is considered good if it performs better than both
Because with Tickbooster it’s possible that you construct your own trading strategy it’s better to have a good understanding of a good or bad simulation:
Here are examples of non-valid backtest simulations:
- Nice yield, but small candidate pool size
- Good yield but trades only on a couple of dates
- Superb yield for all iterations but poor stdev