search

pythonlessons / RL-Bitcoin-trading-bot

Trying to create Reinforcement Learning powered Bitcoin trading bot
MIT License
383 stars 204 forks source link

Lookahead Bias #15

Open MarshallPeng opened 3 years ago

MarshallPeng commented 3 years ago

https://github.com/pythonlessons/RL-Bitcoin-trading-bot/blob/10dac5ef837dd611fe2181e632a1b8b916e0775b/RL-Bitcoin-trading-bot_4/RL-Bitcoin-trading-bot_4.py#L229

Hi Rokas,

Thank you for the lesson and fantastic work on this tutorial. I have a quick question about why you set the current price equal to the 'open' price rather than the Close. The agent is able to see the close price for any time step, but is still able to execute at the open price. Doesn't this provide information that a real trader would not have? In other words, isn't the agent able to see that, for example, the close is higher than the open, and therefore should buy? I think this introduces some form of lookahead bias. Do you know what your results look like if you set the current price to the close?

Thanks

kitmir commented 2 years ago

I do not think this line is look ahead bias. Generally speaking, the closing price of a candle is the opening price of the next candle. But if you want to be sure you can change to: current_price = self.df.loc[self.current_step-1, 'Close'] You will get the same result.

MarshallPeng commented 2 years ago

No, this is most definitely look ahead bias.

At each time step the agent is able to see the closing price, while still being able to buy/sell at the open price. In a real trading scenario, this would never be possible. To make the environment more realistic, you would have to set the current_price = self.df.loc[self.current_step, 'Close']. But, if you try this out, it significantly decreases the performance of the agent compared to what was reported in the tutorial.

Setting current_price = self.df.loc[self.current_step-1, 'Close'] as you suggested does nothing to fix the problem as you are still executing at the previous time step while having knowledge of the next time step.

kitmir commented 2 years ago

I am not sure if I would agree with you but I see what you mean and I will give it another look. But I advice also for you to look into what is being sent to action function. State doesn't include current step. that's the reason why I say it is not lookahead bias.