search

AI4Finance-Foundation / ElegantRL

Massively Parallel Deep Reinforcement Learning. 🔥
https://ai4finance.org
Other
3.62k stars 832 forks source link

Can't run Trifinger environment with Isaac Gym #117

Closed hmomin closed 2 years ago

hmomin commented 2 years ago

When I try to run the Trifinger environment on Isaac Gym using examples/tutorial_Isaac_Gym.py, I receive the following error:

Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/momin/Documents/GitHub/ElegantRL/elegantrl/run.py", line 237, in run
    if_reach_goal, if_save = evaluator.evaluate_save_and_plot(act, steps, r_exp, logging_tuple)
  File "/home/momin/Documents/GitHub/ElegantRL/elegantrl/evaluator.py", line 40, in evaluate_save_and_plot
    rewards_steps_list = [get_episode_return_and_step(self.eval_env, act)
  File "/home/momin/Documents/GitHub/ElegantRL/elegantrl/evaluator.py", line 40, in <listcomp>
    rewards_steps_list = [get_episode_return_and_step(self.eval_env, act)
  File "/home/momin/Documents/GitHub/ElegantRL/elegantrl/evaluator.py", line 141, in get_episode_return_and_step
    state, reward, done, _ = env.step(action)
  File "/home/momin/Documents/GitHub/ElegantRL/elegantrl/envs/IsaacGym.py", line 179, in step
    tensor_state_dict, tensor_reward, tensor_done, info_dict = self.env.step(
  File "/home/momin/Documents/GitHub/ElegantRL/elegantrl/envs/isaac_tasks/base/vec_task.py", line 355, in step
    actions = self.dr_randomizations["actions"]["noise_lambda"](actions)
  File "/home/momin/Documents/GitHub/ElegantRL/elegantrl/envs/isaac_tasks/base/vec_task.py", line 665, in noise_lambda
    corr
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

I'll look into this.

Yonv1943 commented 2 years ago

I solve this bug with adding actor = actor.to(device) in Evaluator. device should be consistent with the GPU id of Isaac Gym for Evaluator.

Evaluator reveices an actor(CPU) from Learner. Then Evaluator evaluate this actor on its env.

It is OK for OpenAI standard gym (such as MuJoCo, PyBullet, etc.) But Isaac Gym require an actor in (GPU), and the GPU id should be consistent with the GPU id of Isaac Gym.

Now I am rewriting the code about Isaac Gym Preview 3 + ElegantRL. I will upload these code later.