search

isaac-sim / IsaacGymEnvs

Isaac Gym Reinforcement Learning Environments
Other
2.07k stars 430 forks source link

Hydra multirun causes Segmentation fault (core dumped) #96

Open generush opened 1 year ago

generush commented 1 year ago

Hello,

Thank you for developing this repo and open sourcing it, it has been very helpful for my own research work.

I recently tried to utilize the Hydra multirun command to conduct a parameter sweep of my deep RL network. However, upon doing that, I received a seg fault, and I am not sure how to resolve it. I hope we can find a solution! If there are more convenient methods of running parameters sweeps instead of using Hydra, I would also be curious to hear your thoughts.

Segmentation fault (core dumped)

The seg fault occurs after calling isaacgymenvs.make and entering _create_env. Typically, the seg fault occurs 50-75% of the way through _create_env, though it seems stochastic.

I continued to investigate this problem, and wanted to know if it was specific to my code or not. So I went to the IsaacGymEnvs repo, and inside the folder IsaacGymEnvs/isaacgymenvs, ran via the terminal:

python train.py -m seed=0,1 max_iterations=10 force_render=False

(this conducts two runs, seed=0 and seed=1, I set the other parameters so it would train the first run within seconds and move quikcly to the second run) and received the same seg fault, this time with an error message:

[Error] [carb.gym.plugin] Function GymGetActorDofStates cannot be used with the GPU pipeline after simulation starts.  Please use the tensor API if possible.  See docs/programming/tensors.html for more info.
Segmentation fault (core dumped)

At this point, I realized it may be a problem within the IsaacGymEnvs code base, and that it's unlikely I could resolve this problem on my own, so I decided to communicate it here. If there is some glaring mistake or misunderstanding I am making, I welcome feedback! Thank you.

zhichenghe2008 commented 1 week ago

I have the same issue in my docker container env, and isaac gym can not working without GUI？ There sitll have same issue when I using headless mode.