Closed TMorville closed 4 years ago
There was an error in the exception loop.
Replacing self.context_dim
with self.latent_dim
yields the correct dimensions and allows training.
Can authors confirm that this is correct? If yes, I can do a PR.
I was able to get the following error after following the reproduction steps -
2 11 891
Started at 31-08-20 14:44:10
Training NeuralLinearPosteriorAgent on TitanicDataBandit for 5000 timesteps
timestep regret/regret reward/reward regret/cumulative_regret reward/cumulative_reward regret/regret_moving_avg reward/reward_moving_avg
100 1 0 51 49 0.51 0.49
200 1 0 94 106 0.47 0.53
300 0 1 140 160 0.448 0.552
400 1 0 195 205 0.492 0.508
500 0 1 229 271 0.44 0.56
Encounterred exception during training!
array must not contain infs or NaNs
Training completed in 1 seconds
Final Regret Moving Average: 0.444 | Final Reward Moving Average: 0.556
Traceback (most recent call last):
File "/content/genrl/genrl/trainers/bandit.py", line 185, in train
action = self.agent.select_action(context)
File "/content/genrl/genrl/agents/bandits/contextual/neural_linpos.py", line 128, in select_action
for i in range(self.n_actions)
File "/content/genrl/genrl/agents/bandits/contextual/neural_linpos.py", line 128, in <listcomp>
for i in range(self.n_actions)
File "mtrand.pyx", line 4082, in numpy.random.mtrand.RandomState.multivariate_normal
File "/usr/local/lib/python3.6/dist-packages/scipy/linalg/decomp_svd.py", line 109, in svd
a1 = _asarray_validated(a, check_finite=check_finite)
File "/usr/local/lib/python3.6/dist-packages/scipy/_lib/_util.py", line 246, in _asarray_validated
a = toarray(a)
File "/usr/local/lib/python3.6/dist-packages/numpy/lib/function_base.py", line 499, in asarray_chkfinite
"array must not contain infs or NaNs")
ValueError: array must not contain infs or NaNs
This was resolved by removing nans from the dataframe through a self._df = self._df.fillna(0)
The reason the error shows up after 500 steps is that by default, the DCBTrainer
waits 500 steps before starting to update the agent's parameters.
There was an error in the exception loop.
Replacing
self.context_dim
withself.latent_dim
yields the correct dimensions and allows training.Can authors confirm that this is correct? If yes, I can do a PR.
This might also be a legitimate issue. Looking into it
@TMorville could you check if removing nan values from the dataframe works for you?
I dont get the NaN error when running the code 🤔
1) If I keep nan values in data and use 'latent_dim' it also works and leads to a cumulative reward of 1843.
2) If I remove nan values in data and use context_dim
it also works and leads to a cumulative reward of 1539.
Yep, you are right, it looks like it should be latent_dim
instead of context_dim
.
Not sure why the except
block was being executed in your case since it wasnt in mine. Maybe different numpy
versions might calling different things LinAlgError
. 🤔
Either way, thanks for raising the issue! Feel free to open a PR.
- If I keep nan values in data and use 'latent_dim' it also works and leads to a cumulative reward of 1843.
Is this after running for 5000 steps? I am consistently getting >3000 cumulative reward.
Played around with it for a bit and found that:
SVD did not converge
, which then fails because of the typo.My genrl is installed from source with version 0.0.1.
I am getting an error that might be related to #300, or possibly to do with custom implementation.
To reproduce:
1) Make a user and download the data here: https://www.kaggle.com/c/titanic 2) Running this code creates as genrl compatible data set
Now run the custom bandit code with the small change from #300 (still to be confirmed if actually an error):
yields a shape error:
I can't seem to figure out why this fails after 500 time steps.
EDIT:
Seems that the shape of
beta
is changed from [2, 51] to [2, 12] at timestep 500. For some reason this part ofNeuralLinearPosteriorAgent.select_action
failsand the exception occurs triggering a new calculation of beta: