News and updates: Slack channel added

[Kismuz]

vvv Scroll down for latest updates vvv

• 6.02.18: Pushed common update to all a3c agents architectures:
  • all dense layers are now Noisy-Net ones, see: Noisy Networks for Exploration paper by Fortunato at al.;
    • note that entropy regularization is still here, lowered to ~0.01 to ensure proper exploration; update your entropy_beta custom settings, if done any;
  • policy output distribution is 'centered' using layer normalisation technique;
  • added per-channel weights scaling to obs state before passing to tanh, see #35 and https://github.com/Kismuz/btgym/blob/master/examples/state_signal_scaling.ipynb

All of the above results in about 2x training speedup in terms of train iterations: stacked LSTM agents takes ~20M steps to converge on 6 month real data set (was: 40).

[Kismuz]

- 17.02.18: First results on applying guided policy search idea (GPS) to btgym setup: https://github.com/Kismuz/btgym/blob/master/examples/guided_a3c.ipynb

Documentation on GPS API: https://kismuz.github.io/btgym/btgym.research.gps.html

- tensorboard summaries are updated with additional renderings: actions distribution, value function and LSTM_state; presented in the same notebook.

[Kismuz]

• 20.07.2018: major update to package:

  • enchancements to agent architecture:

    • casual convolution state encoder with attention for LSTM agent;
    • dropout regularization added;

  • strategy: new convention for naming get_state methods, see BaseStrategy class for details;

  • multiply datafeeds and assets trading implemented in two flavors:

    • discrete actions space via MultiDiscreteEnv class;

    • continious actions space via PortfolioEnv which is closely related to contionious portfolio optimisation problem setup;

      • description and docs:

        • MultiDataFeed: https://kismuz.github.io/btgym/btgym.datafeed.html#btgym.datafeed.multi.BTgymMultiData
        • ActionSpace: https://kismuz.github.io/btgym/btgym.html#btgym.spaces.ActionDictSpace
        • MultiDiscreteEnv: https://kismuz.github.io/btgym/btgym.envs.html#btgym.envs.multidiscrete.MultiDiscreteEnv
        • PortfolioEnv: https://kismuz.github.io/btgym/btgym.envs.html#btgym.envs.portfolio.PortfolioEnv

      • examples:

        • MultiDiscreteEnv: https://github.com/Kismuz/btgym/blob/master/examples/multi_discrete_setup_intro.ipynb
        • PortfolioEnv: https://github.com/Kismuz/btgym/blob/master/examples/portfolio_setup_BETA.ipynb
    • Notes on multi-asset setup:
      • adding these features forced substantial package redesign; expect bugs, some backward incompatibility, broken examples etc - please report;
      • current algorithms and agents architectures are ok with multiply data lines but seem not to cope well with multi-asset setup. It is especially evident in case of continuous actions, where agents completely fail to converge on train data;
      • current reward function design seems inappropriate; need to reshape;
      • continuous space in beta and still needs some improvement, esp. for broker order execution logic as well as action sampling routine for continuous A3C (which is Dirichlet process by now);
      • multi-discrete space is more consistent but severely limited in number of portfolio assets (but not data-lines) due to exponential rise of action space cardinality; the option is to as use many datalines as desired while limiting portfolio to 1 - 4 assets;
      • no Guided Policy available for multi-asset setup yet - in progress;
      • all but episode rendering modes are temporally disabled;
      • whole thing is shamelessly resource-hungry;

[Kismuz]

• 14.10.2018: update: base reward function redesign
  • two aux. reward potential functions excluded ( -> reward bias removed);
  • state potential function f1 computation logic redesigned making estimation less noisy and more consistent;
  • results in noticeable performance gain; skip_frame parameter now can be set as low as 2 frames per action (was: ~10 for stable convergence).

[Kismuz]

• 11.12.2018: updates and fixes:

  • training Launcher class got convenience features to save and reload model parameters, see https://github.com/Kismuz/btgym/blob/master/examples/unreal_stacked_lstm_strat_4_11.ipynb for details
  • combined model-based/model-free approach package in early development stage is added to btgym.reserach

• 17.11.2018: updates and fixes:

  • minor fixes to base data provider class episode sampling update to btgym.datafeed.synthetic subpackage: new stochastic processes generators added etc. new btgym.research.startegy_gen_5 subpackage: efficient parameter-free signal preprocessing implemented, other minor improvements

[Kismuz]

• 18.01.2019: updates:

  • data model classes are under active development to power model-based framework:

    • common statistics incremental estimator classes has been added (mean, variance, covariance, linear regression etc.);
    • incremental Singular Spectrum Analysis class implemented;
    • for a pair of asset prices, two-factor state-space model is proposed

  • new data_feed iterator classes has been added to provide training framework with synthetic data generated by model mentioned above;

  • strategy_gen_6 data handling and pre-processing has been redesigned:

    • market data SSA decomposition;
    • data model state as additional input to policy
    • variance-based normalisation for broker statistics

[Kismuz]

• 25.01.2019: updates:
  • lstsm_policy class now requires both internal and external observation sub-spaces to be present and allows both be one-level nested sub-spaces itself (was only true for external); all declared sub-spaces got encoded by separate convolution encoders;
  • policy deterministic action option is implemented for discrete action spaces and can be utilised by syncro_runner; by default it is enabled for test episodes;

[Kismuz]

• 9.02.2019:
  • Introduction to analytic data model notebook added to model_based_stat_arb examples folder.

[Kismuz]

• 24.02.2019:
  • Public Slack channel added. Join here.