Regression issue with models on latest BTGym version

[JaCoderX]

@Kismuz I've been playing with a few variation of a model I've created since about gen 5 was published.

got some good (and fast) convergence for the simple sin data (~3000-5000 steps depended on what variation i was testing).

Then I got a few setbacks when encountered #95, which by this time I have updated the repo to the new gen 6 update.

I had retested the models that showed good results before both on gen 6 and 5 but I cannot reproduce the results anymore under the new code.

*I have kept the old results in tensorboard for comparison

not sure on how to continue from here, don't really want to revert the code and work with old version but new version doesn't give me the same results (not even close)

I'm just posting my latest experience, a bit general but I'm not sure what cause the big difference

[Kismuz]

@JacobHanouna , there are several changes has been made to btgym which can affect performance:

• changed iteration order, see #84 (can't really cause performance degradation)
  • startegy gen_6: new normalisation algorithm based on exponentially smoothed rolling asset volatility has been applied to all broker statistic and reward estimation; this change targets statistical arbitrage setup (with more ore less stationary synthetic spread as trading asset) and has not been tested with non-stationary assets (equity or currence prices); this code has undergone several principal changes to get it dedicatedly working with arbitrage setup; think it is better to stick with gen_4 routine for later cases.
  • gen_6 also uses simple heuristics to compute dynamic order size based on current account value, asset price and preset maximum exposure depth;

Anyway, if you can temporarily unroll the code and compare differences with current version (say, via Git) it can be beneficial to understand what caused performance drop.

[JaCoderX]

Anyway, if you can temporarily unroll the code and compare differences with current version (say, via Git) it can be beneficial to understand what caused performance drop.

@Kismuz it took me a while to unroll and run enough revisions to identify the commit that show first sign of the regression issue.

setup description:

• same code was used for all checked revisions (unless needed to change dim in strategy due to code change)
• BTgymMultiData is being used for multi data support in strategy
• I've created a custom strategy for handling multi data based on MonoSpreadOUStrategy_0
• All data used in evaluation were simple sin data from the example folder
• revision that were checked (555, 586, 591, 593, 594, 595, 598, 599, 600)

results:

• rev 593 was the last rev to show good results
• rev 591 had the best results (rev 592 wasn't checked)
• all rev since 594 show very poor results

conclusions:

• at rev 593 MonoSpreadOUStrategy_0 was based on gen 6, so issue is not related to strategy gen 6
• possible bad conflict merge at rev 594 - 'branch conflict merge commit'

@Kismuz if any other info is needed I have the tensorboard result for each revision.

also if possible and accepted by you it would be nice if you can put back in the code MonoSpreadOUStrategy_0. I find it to be a very good and stable base strategy especially because of changes made to get_external_state

[Kismuz]

@JacobHanouna, sorry for very late reply. After a quick sweep I can't replicate mentioned performance degradation. Can you please post TB summaries here?

MonoSpreadOUStrategy_0

I'll look for that code.

[JaCoderX]

rev 593:

rev 594:

from rev 594 onward the summaries give more or less same results. and before rev 593 it show consistent results.

I also tried letting some of the revisions after 594 run for ~20 K but result stay the same

[JaCoderX]

I retested this issue after a few design changes I made in my models. Error doesn't seem to reoccur.

Still not sure what caused it in the first place, but it might have been model related