Ceruleanacg / Personae

📈 Personae is a repo of implements and environment of Deep Reinforcement Learning & Supervised Learning for Quantitative Trading.
MIT License
1.34k stars 338 forks source link

It seems that the RL algorithms under default parameters didn't learn anything. #22

Closed xtyangjie closed 6 years ago

xtyangjie commented 6 years ago

Another question(It can be found in my mail, too) is as shown in the title. I am puzzled whether I had misunderstanded. Please help me to realize it...

I ran 3 alogrithms of NaiveLSTM, TreNet and Attention. After training, the predictions were plotted. While I made a tiny change on the plotting method - I added the close price at time t in x, to compare it with real and predict. Then in the figure, there were now 3 lines:

What puzzled me is the predict line approximated much more with x instead of with real in the result of all the three algorithms. In my former experiments, I used some form of simple functions such as sin(x) as a sequence, and plotted x, real, and predict. On these simple sequences the predict lines approximate the real(y, time t+1) perfectly.

As a result, I have got a question whether the algorithms learned anything. Please give me some help...

Following 3 plots are NaiveLSTM, the whole plot of Attention, and a local figure in the plot of Attention. In all of which, there 3 lines indicating x(just close price at time t, real, and predict).

  1. NaiveLSTM image
  2. Whole plot of Attention (In which the detail is hard to distinct) image
  3. A local figure of Attention 3

Thanks very much!

Ceruleanacg commented 6 years ago

We had a talk in WeChat.