nateemma / strategies

Custom trading strategies using the freqtrade framework
329 stars 88 forks source link

Persistent model storage and signal prediction accuracy #12

Open just-nilux opened 1 year ago

just-nilux commented 1 year ago

@nateemma thank you so much for your repo - I'm working on my modified PCA strategies for a few weeks now with great results. I really enjoy diving deeper into all this.

Two questions:

If you could point me in the right direction I would appreciate that! Keep up the good work

nateemma commented 1 year ago

some answers below...

On Wed, Feb 22, 2023 at 9:15 AM nilux @.***> wrote:

@nateemma https://github.com/nateemma thank you so much for your repo - I'm working on my modified PCA strategies for a few weeks now with great results. I really enjoy diving deeper into all this.

Two questions:

-

Currently the model of PCA is stored in memory right? Wouldn't it make more sense to use a persistent storage on disk to retain models even after restart? Like it's done in freqai. Is there a way to change this to disk storage?

It sort of does this already. When I run in backtesting mode, the resulting models (one per pair) are saved to disk. In other modes, those models are reloaded and used to initialise the classification algorithms, but I then re-fit using 'live' data. I haven't tried using just the saved model, but will try that when I have some time. FYI, the underlying (sklearn) classifiers are not cumulative, so I can't incrementally train them Regarding memory, you have to have the data in memory to run the classifiers anyway so saving them to disk doesn't really help.

-

I would like to incorporate the accuracy of predictions into my risk management. Basically check if the past x predictions lead to a profitable trade. How would I approach this to get such accuracy score of the recent buy signals and recent sell signals?

If you set the flags dbg_verbose and dbg_test_classifier to True, then you will see accuracy scores (using test data, not training data). If you want to re-use that code, look at train_models(0 (in PCA.py), at the end of the function (or just search for if self.dbg_test_classifier and self.dbg_verbose:

-

If you could point me in the right direction I would appreciate that! Keep up the good work

Hope that helps

Cheers,

Phil

— Reply to this email directly, view it on GitHub https://github.com/nateemma/strategies/issues/12, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABD4X56TYV54UYVJGEFCMTLWYZCTBANCNFSM6AAAAAAVETJGB4 . You are receiving this because you were mentioned.Message ID: @.***>

just-nilux commented 1 year ago

Thank you Phil! That helps. I will look into this.

just-nilux commented 1 year ago

Re model storage: I think you misunderstood me... the NN* strats have model files, that's obvious, but the PCA doesn't. I couldn't find any model files.

Don't you think it would make sense to have the PCA models stored on disk for persistence?

nateemma commented 1 year ago

Huh, guess I took that out. When I have some time I'll take another look at that. It might not be possible, due to the varying nature of the dimensionality reduction step (it may change over time), but I guess I could just reduce dimensions to a fixed size (I already do this in the NN strategies because the dimensions have to be fixed)

Thanks,

Phil

On Wed, Feb 22, 2023 at 10:19 AM nilux @.***> wrote:

Re model storage: I think you misunderstood me... the NN* strats have model files, that's obvious, but the PCA doesn't. I couldn't find any model files.

Don't you think it would make sense to have the PCA models stored on disk for persistence?

— Reply to this email directly, view it on GitHub https://github.com/nateemma/strategies/issues/12#issuecomment-1440558038, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABD4X566JZR6FH6LH7W727LWYZKEZANCNFSM6AAAAAAVETJGB4 . You are receiving this because you were mentioned.Message ID: @.***>

just-nilux commented 1 year ago

Yep, some persistence would be great. I noticed the performance and predictions vary A LOT after every restart/reload of freqtrade even with same settings. I guess thats just the because the mode lis being retrained on new data after each restart.