updated-NES-google.ipynb 0 reward

[ghost]

Hi, Thank you for sharing your project. I'm trying to run updated-NES-google.ipynb(converted to py) with a crypto data I attached and I'm getting this output with no rewards so no buy-sell signals. Everything works fine with yahoo stock data including stocks and other crypto currency data in yahoo but when I put any data with BTC as the opposite currency(more than one 0 digits on the right side) the below outpus happens; it should be abs/division error something but could not figure out where it is. do you have an idea why?

RuntimeWarning: invalid value encountered in true_divide rewards = (rewards - np.mean(rewards)) / np.std(rewards) iter 10. reward: 0.000000 iter 20. reward: 0.000000 iter 30. reward: 0.000000 iter 40. reward: 0.000000 iter 50. reward: 0.000000 iter 60. reward: 0.000000 iter 70. reward: 0.000000 iter 80. reward: 0.000000 iter 90. reward: 0.000000 iter 100. reward: 0.000000 iter 110. reward: 0.000000 iter 120. reward: 0.000000 iter 130. reward: 0.000000 iter 140. reward: 0.000000 iter 150. reward: 0.000000 iter 160. reward: 0.000000 iter 170. reward: 0.000000 iter 180. reward: 0.000000 iter 190. reward: 0.000000 iter 200. reward: 0.000000 iter 210. reward: 0.000000 iter 220. reward: 0.000000 iter 230. reward: 0.000000 iter 240. reward: 0.000000 iter 250. reward: 0.000000 iter 260. reward: 0.000000 iter 270. reward: 0.000000 iter 280. reward: 0.000000 iter 290. reward: 0.000000 iter 300. reward: 0.000000 iter 310. reward: 0.000000 iter 320. reward: 0.000000 iter 330. reward: 0.000000 iter 340. reward: 0.000000 iter 350. reward: 0.000000 iter 360. reward: 0.000000 iter 370. reward: 0.000000 iter 380. reward: 0.000000 iter 390. reward: 0.000000 iter 400. reward: 0.000000 iter 410. reward: 0.000000 iter 420. reward: 0.000000 iter 430. reward: 0.000000 iter 440. reward: 0.000000 iter 450. reward: 0.000000 iter 460. reward: 0.000000 iter 470. reward: 0.000000 iter 480. reward: 0.000000 iter 490. reward: 0.000000 iter 500. reward: 0.000000 time taken to train: 360.73433113098145 seconds

when I debug the line it warns I see that everything is 0. It is always 0...

-> rewards = (rewards - np.mean(rewards)) / np.std(rewards) (Pdb) rewards array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]) (Pdb) rewards array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])

Archive.zip

[neoricalex]

Here also with the original google csv and code

[UPDATE] Found! You need to raise the layer_size

[huseinzol05]

Closed? you really need to plot the trend first before train the model.

[neoricalex]

Hi @huseinzol05 , first, thanks for youre work.

When you find some minutes, can you please elaborate how the layer_size, window_size, population_size and the CSV size play together?

i know that is tricky, but its only for that i can grasp the general idea. i found that of raise the layer size but i dont have any idea why :-s

Many thanks !

[huseinzol05]

If these parameters pretty hard to play around, you can let AI choose for you! There is an agent called bayesian agent sit on top of NES agent, his job only need to find good parameters for your own timeseries dataset. You can check it here, bayesian-agent

[huseinzol05]

There is no exact answer about NES parameters, NES is using neural network, if you are asking me about why increasing layers can improve the result, I also cannot answer that. This is the problem about non-convex optimization

[neoricalex]

Yes tricky indeed. Only to be sure that's not me rnrn

Thanks again for all :-)

[ghost]

ok, thank you. This happened because I commented out the initial plots

[ghost]

it's not about the plots, this error comes up in every file in agent directory.. this one:

RuntimeWarning: invalid value encountered in true_divide
rewards = (rewards - np.mean(rewards)) / np.std(rewards)

and this one:

Evolution_Bayesian.py:66: RuntimeWarning: invalid value encountered in true_divide
  rewards = (rewards - np.mean(rewards)) / np.std(rewards)
stop after 100 iteration with investment 0.000000
|  250      |  0.0      |  0.49     |  34.42    |  0.01     |  628.3    |  15.0     |  17.67    |
=================================================================================================
----------------------------------------------
{'target': 0.005529999999998836, 'params': {'learning_rate': 0.07725429450379967, 'population_size': 27.42513865462102, 'sigma': 0.3545970562206382, 'size_network': 539.0312796221687, 'skip': 3.221331899516282, 'window_size': 13.424744381086434}}
Traceback (most recent call last):
  File "Evolution_Bayesian.py", line 325, in <module>
    agent.fit(500, 100)
  File "Evolution_Bayesian.py", line 172, in fit
    self.es.train(iterations, print_every = checkpoint)
  File "Evolution_Bayesian.py", line 65, in train
    rewards[k] = self.reward_function(weights_population)
  File "Evolution_Bayesian.py", line 146, in get_reward
    action, buy = self.act(state)
  File "Evolution_Bayesian.py", line 136, in act
    return np.argmax(decision[0]), int(buy[0])
ValueError: cannot convert float NaN to integer