search

edtechre / pybroker

Algorithmic Trading in Python with Machine Learning
https://www.pybroker.com
Other
2.09k stars 261 forks source link

profit_factor below confidence interval #73

Closed saddy001 closed 1 year ago

saddy001 commented 1 year ago

When using result = strategy.backtest(calc_bootstrap=True, train_size=0.5) I often see the result.metrics.profit_factor being lower than result.bootstrap.profit_factor.low_2p5. I see it more often than one would expect from a 2.5% chance.

I have two questions:

  1. Can you confirm this (without a minimal reproducing example)?
  2. Is boostrapping in combination with the train_size parameter even supported?
edtechre commented 1 year ago

low_2p5 does not represent a 2.5% chance. It represents a 97.5% confidence interval.

For #2, bootstrapping samples from the test splits. The train_size param specifies how large the train/test splits are.