apply deep Q network on stock candlestick and MA

[musiclicn]

input: state: past N days prices including OHLC, 5 MA, 10 MA, 20 MA, 50 MA and 200 MA (including position) dollar amount action: NA (no action), buy 1/3, buy 2/3, buy all, sell 1/3, sell 2/3, sell all reward: discounted future 5 day PNL

Notes: use CNN for policy function/ network without pooling

[musiclicn]

MileStone:

1. set up vnpy and run backtesting
2. install OpenAI/baselines, gym (there are alternatives of dqn libraries other than baselines https://libraries.io/github/devsisters/DQN-tensorflow ) . https://github.com/openai/gym/tree/master/gym/envs
3. run dqn examples and understand it
4. play with the dqn, dueling-dqn
5. create customized trade environment for backtesting by integrate vnpy https://github.com/openai/gym/blob/master/gym/envs/toy_text/blackjack.py https://stackoverflow.com/questions/44469266/how-to-implement-custom-environment-in-keras-rl-openai-gym
6. use dueling-dqn to train trade-bot with trade env

[musiclicn]

reward function:

1. PNL
2. PNL with drawdown penalty
3. sharp ratio?

[musiclicn]

apply latest advanced RL algorithm https://blog.openai.com/baselines-acktr-a2c/

[musiclicn]

Reward function = average sharp ratio (or volatility) of next N period reward +1 when sharp ratio increase; -1 when sharp ratio decrease

[musiclicn]

observation contains position level, past N prices and indicators