saeed349 / Deep-Reinforcement-Learning-in-Trading

This repository provides the code for a Reinforcement Learning trading agent with its trading environment that works with both simulated and historical market data. This was inspired by OpenAI Gym framework.
207 stars 86 forks source link

Questions on minor discrepancies w/ paper #4

Open ebagjones opened 4 years ago

ebagjones commented 4 years ago

This is a great project and I like what has been done here, but I've been going through the code and paper and have some questions.

  1. I can't find TP, SL descriptions. To Purchase, SelL?
  2. Trading fees are excluded when buying, selling at the same price - fractionally, a large number of positions are opened, closed at the same trade price, but this doesn't account for fees or slippage. This can be significant drag on the PnL.
  3. Just a note, your Histograms have a bug, the Buy and Hold and DDDQN labels are swapped. I think this error might have even carried into your paper.

Thanks again (and in advance!)

lorrp1 commented 4 years ago

the model has many problems.

it seems to always converge to a strategy in which some position can remain open even with long-term drawback. even giving a smaller reward based on the self.unreturn the model never change unless it stop opening position (which become the only way to avoid massive drawback) lowering the reward when its position flat + counting drawback the model never change.

also i dont understand how is that possible that the results with low episode are the same with higher episode, i batch size as well with little to no results.

tp sl are used to render closed position.

ebagjones commented 4 years ago

Lor, That makes a lot of sense, I've been working with the stop loss, profit taking, and reward and it doesn't seem to be working as expected. Are you considering digging into these issues, or moving on to something else?

lorrp1 commented 4 years ago

using predefined stop losses and take profit would make it worse.

probably no since i would not know where to start to fix this problem