gym_forward_walker.py:50: RuntimeWarning: invalid value encountered in greater

[DianaKessler]

I reinstalled linux on my pc and now I get the following error when running RoboschoolHalfCheetah_v0_2017may.py - error that I did not encounter before.

I will print the output in the following lines:

 Making new env: RoboschoolHalfCheetah-v1
 score=2674.77 in 1000 frames
 PATH_TO_ROBOSCHOOL/roboschool/gym_forward_walker.py:50: RuntimeWarning: invalid value encountered in greater
   self.joints_at_limit = np.count_nonzero(np.abs(j[0::2]) > 0.99)
 ~INF~ [nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan
  nan nan  0.  0.  0.  0.  0.  0.]
 score=nan in 1 frames
 Traceback (most recent call last):
   File "PATH_TO_ROBOSCHOOL/agent_zoo/RoboschoolHalfCheetah_v0_2017may.py", line 281, in <module>
     demo_run()
   File "PATH_TO_ROBOSCHOOL/agent_zoo/RoboschoolHalfCheetah_v0_2017may.py", line 33, in demo_run
     obs, r, done, _ = env.step(a)
   File "path_to_gym/gym/core.py", line 96, in step
     return self._step(action)
   File "path_to_gym/gym/wrappers/time_limit.py", line 36, in _step
     observation, reward, done, info = self.env.step(action)
   File "path_to_gym/gym/core.py", line 96, in step
     return self._step(action)
   File "PATH_TO_ROBOSCHOOL/roboschool/gym_forward_walker.py", line 91, in _step
     self.apply_action(a)
   File "PATH_TO_ROBOSCHOOL/roboschool/gym_forward_walker.py", line 41, in apply_action
     assert( np.isfinite(a).all() )
 AssertionError

Could you please give me some suggestions on how I can solve this issue?

[ionalina]

This is an interesting question but I am afraid I don't have an answer. And it triggers me too. Did you manage to solve it?

[psanketi]

This indicates some sort of numerical instability. What is it you are trying to do?

For me, this happened to me recently when I was using a tf variable for representing std of a normal distribution. I was sampling from that distribution and calculating prob of chosen actions in a RL setting. After changing the tf Variable to represent the log of the std, the error went away. Basically, my normal distribution was sy_logstd = tf.Variable(0.0, name="sy_logstd") # logstd instead of std dist = tf.distributions.Normal(loc = sy_mean, scale = tf.exp(sy_logstd)) # note tf.exp()). So check if you have any such variables, and use log instead.

[DianaKessler]

@psanketi Thank you for the answer. I am just trying to run Roboschool_HalfCheetah_v0 as it is provided on github. The error happens at the beginning of the second iteration. The problem is that I have it installed on an external hard drive and it works. I have the error on the actual pc (specify that I installed it using the same commands as I did on the external hard drive).

So, I did not change anything from the provided code on the actual pc.

[AlEmerich]

Hi, @DianaKessler

Did you manage to solve that problem ? I have the same one with RoboschoolHumanoid-v1, which I referenced in #132 and it is the same than you, I have that error after the second reset.

[tiagokv]

I'm running RoboschoolHumanoidFlagrun_v0_2017may.py and is also occurring the same error. In which OS/Distro are you running?

[AlEmerich]

Ubuntu 16.04 for me

[AhmedMagdyHendawy]

@DianaKessler @AlEmerich anyone reached to the solution ?

[AlEmerich]

@AhmedMagdyHendawy Nope... In order to be able to use that environment, I am training my agents without render because I have no problems at the reset when I am not rendering the env.

In order to still see how the training is going, I plot some of my relevant metrics with matplotlib dynamically by updating the value in the plot at every epochs (for RoboschoolHumanoid, I plot reward, distance gravity center from the ground, distance to the target and angle to the target).

Just a precision: I am using Ubuntu16.04 and the computer at my work is a Mac. I tested on this last one and I had no problem. So maybe this a problem of Bullet implementation which is different from each distro, a synchronisation problem I guess.

[Frawak]

I have the same problem. This AssertionError is thrown at each environment reset when rendering. I just want to add some observations encountering this problem: 1) It definitely occurs at the first taken step after the reset. Running one rendered episode is fine. 2) I loaded the Roboschool reacher and pong environment in the random-agent script of the gym repository, removed the monitor and added just a env.render("human") after the step. After the reset, the display window freezes and the following line is spammed in the terminal: QImage::setPixel: coordinate [some coordinates] out of range 3) Point 2 occurs at rendering after the step, i.e. the original AssertionError is not thrown taking the step.

...maybe this gives someone an idea. I am using Ubuntu16.04 as well.

[FitMachineLearning]

@AlEmerich ,

Could you share extracts of code showing how you retrieve distance and other metrics you plot in matplotlib.

FitML

[AlEmerich]

Sure @FitMachineLearning ,

Here is the object who manage the plot: Board

And here is the object who launch the data to it Abstract env

I were in bad place to achieve to do something dynamyc, don't hesitate to tell me if you find bug

[FitMachineLearning]

Thanks a bunch Allan.

Just what I was looking for.

[Schakirov]

I have the same problem on both Ubuntu 16.04 and Ubuntu 18.04.

[timokau]

This has been happening for a while, more analysis by @denizs in https://github.com/openai/roboschool/issues/7#issuecomment-333171358.

[denizs]

Back in 2017, I started to dig into this issue, but it seems to be burried somewhere in the physics engine. Anybody keen to help solving this issue?