search

llSourcell / Reinforcement_Learning_for_Stock_Prediction

This is the code for "Reinforcement Learning for Stock Prediction" By Siraj Raval on Youtube
638 stars 362 forks source link

evaluate.py #6

Open yeshengyi opened 6 years ago

yeshengyi commented 6 years ago

The training works fine, but evaluate always gives profit=0, where is the model.fit?

Seems like the model is only compiled, but not fitted...

rajvanshyr commented 6 years ago

I'm getting same issue, going to look into the blog that he posted to see if I can find a way to fix it. If anyone else finds a way to fix it please let me know.

bartimusprimed commented 6 years ago

I believe this is my mistake, in the agent/agent.py the random.random() is supposed to be np.random.rand()

bartimusprimed commented 6 years ago

i will check and update

xtr33me commented 6 years ago

Getting the same thing....first wanted to check here to see if anyone had posted an issue already. Going to take a look myself to see what may be causing it.

xtr33me commented 6 years ago

I found the problem to be that the original logic this was based on expected randomness to occur from the start and then the model will pick up once epsilon met a certain criteria, however in evaluate we never run expReplay, nor do I believe we have to, however because of this a "Buy" never occurs. So for now the simple solution to get the model running in eval was to issue a buy during the first iteration through, then the model runs fine the rest of the way. Perhaps a change to this in a future implementation will be to find the approximate best time to enter/buy the stock and then from there let the model run.

This was my analysis for the night. Hope it helps some of you get going again. You can find the changes at the below pull request or my fork. It's a bit late here so headin to bed.

https://github.com/llSourcell/Reinforcement_Learning_for_Stock_Prediction/pull/7

yeshengyi commented 6 years ago

the fix works well, thanks!

MonaTanggg commented 6 years ago

I tried that fix, for my try, it only buy at the first time, then the program end. @yeshengyi You used his latest commit and your model work fine ?

yeshengyi commented 6 years ago

Sorry, I was too quick to say that.. The profit is no longer 0, but it was all Sell after the first buy...

xtr33me commented 6 years ago

Hmmm...wondering if something happened during the merge...one sec let me try on mine.

xtr33me commented 6 years ago

Alright....so it was the case. I made a backup last night before performing the merge and missed 2 lines at the bottom of evaluate.py. Can one of you please try the below additional call and let me know if it works for you. I will add to pull request after verification. Sorry about that.

https://github.com/xtr33me/Reinforcement_Learning_for_Stock_Prediction/commit/0cdba092b42091143543aa532eb975e269d928ba

yeshengyi commented 6 years ago

It really works this time... Thanks!

xtr33me commented 6 years ago

Thanks @yeshengyi for testing. Will add to PR.

Draichi commented 6 years ago

working good here too, thanks @xtr33me :smile:

edwin-yan commented 6 years ago

@xtr33me I noticed this too, so I play around the code. I may be wrong, just my 2 cents. I feel this is due to the design of the model. During the training, the sit, buy, sell activity is randomly generated, so there is no problem. However, the reward only tie to sell, so there is no reward to buy or sell, because there is no linkage between other actions and sell. Therefore, the prediction is always sell. Even with buying in the first iteration, it will just sell the 2nd day. Then it kept predicting sell, while there is no inventory at all.