How to display the robot training GUI in PyBullet

[arduino183454]

Hello, I reproduced your code, but I can't see the robot during training. If I want to display the robot training process( PyBullet GUI ) , How should I modify this code? Thank you very much.

[arduino183454]

    I try to modifie  this code, but an error was reported. Looking forward to your help, thank you very much.
    self.train_env = gym.make(self.cfg.task, render_mode='human')
    self.eval_env = gym.make(self.cfg.task, render_mode='human')

[TaoHuang13]

Hello. The visualization of training has been integrated into WandB. You can turn on WandB as instructed in README to see that.

It seems your code is correct, may I know more details about the error? (I guess you should only render train_env with 'human', because render 'human' with 2 envs may result in errors)

[arduino183454]

Thank you very much for your answer，the following is the error when running. Error executing job with overrides: ['seed=3', 'task_name=NeedlePick-v0'] Traceback (most recent call last): File "train.py", line 174, in main exp = Experiment(cfg) File "train.py", line 22, in init self.setup(cfg) File "train.py", line 42, in setup self.eval_env = gym.make(self.cfg.task_name, render_mode='human') File "/home/pan/anaconda3/envs/herdemo/lib/python3.8/site-packages/gym/envs/registration.py", line 597, in make env = cls(**_kwargs) File "/home/pan/强化学习应用/hindsight-experience-replay-with-demo/surrol/tasks/psm_env.py", line 61, in init super(PsmEnv, self).init(render_mode) File "/home/pan/强化学习应用/hindsight-experience-replay-with-demo/surrol/gym/surrol_env.py", line 39, in init self.cid = p.connect(p.GUI) pybullet.error: Only one local in-process GUI/GUI_SERVER connection allowed. Use DIRECT connection mode or start a separate GUI physics server (ExampleBrowser, App_SharedMemoryPhysics_GUI, App_SharedMemoryPhysics_VR) and connect over SHARED_MEMORY, UDP or TCP instead.

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace. Exception ignored in: <function SurRoLEnv.del at 0x7f01d80ec040> Traceback (most recent call last): File "/home/pan/强化学习应用/hindsight-experience-replay-with-demo/surrol/gym/surrol_env.py", line 238, in del self.close() File "/home/pan/强化学习应用/hindsight-experience-replay-with-demo/surrol/gym/surrol_env.py", line 142, in close if self.cid >= 0: AttributeError: 'NeedlePick' object has no attribute 'cid' numActiveThreads = 0 stopping threads Thread with taskId 0 exiting Thread TERMINATED destroy semaphore semaphore destroyed destroy main semaphore main semaphore destroyed finished numActiveThreads = 0 btShutDownExampleBrowser stopping threads Thread with taskId 0 exiting Thread TERMINATED destroy semaphore semaphore destroyed destroy main semaphore main semaphore destroyed

[arduino183454]

During the training process, I cannot see the robot. How should I modify the code? Thank you very much. In this code "https://github.com/TaoHuang13/hindsight-experience-replay-with-demo" I encountered the same problem when run code "sh scripts/run_herdemo.sh " . Thank you very much.

[arduino183454]

Hello, when I run "python3 train.py task=NeedlePick-v0 agent=dex use_wb=True" I can see the robot in the software use_wb.

If I don't use Wb，run "python3 train.py task=NeedlePick-v0 agent =dex use_wb=False”, In this case how to see robots in Pybullet. How to modify this code? Thank you very much。

[TaoHuang13]

Hi. Could you try self.train_env = gym.make(self.cfg.task, render_mode='human'); self.eval_env = gym.make(self.cfg.task, render_mode='rgb_array')?

[arduino183454]

I try self.train_env = gym.make(self.cfg.task, render_mode='human'); self.eval_env = gym.make(self.cfg.task, render_mode='rgb_array'), but it doesn't seem to work, the error is reported as follows. Thanks a lot.

(dex) pan@pan-Lenovo-Legion-Y9000K2021H:~/强化学习应用/hindsight-experience-replh-demo/DEX-main$ python3 train.py task=NeedlePick-v0 agent=sac use_wb=Fals e pybullet build time: Nov 28 2023 23:51:11 startThreads creating 1 threads. starting thread 0 started thread 0 argc=2 argv[0] = --unused argv[1] = --start_demo_name=Physics Server ExampleBrowserThreadFunc started X11 functions dynamically loaded using dlopen/dlsym OK! X11 functions dynamically loaded using dlopen/dlsym OK! Creating context Created GL 3.3 context Direct GLX rendering context obtained Making context current GL_VENDOR=NVIDIA Corporation GL_RENDERER=NVIDIA GeForce RTX 3070 Laptop GPU/PCIe/SSE2 GL_VERSION=3.3.0 NVIDIA 530.41.03 GL_SHADING_LANGUAGE_VERSION=3.30 NVIDIA via Cg compiler pthread_getconcurrency()=0 Version = 3.3.0 NVIDIA 530.41.03 Vendor = NVIDIA Corporation Renderer = NVIDIA GeForce RTX 3070 Laptop GPU/PCIe/SSE2 b3Printf: Selected demo: Physics Server startThreads creating 1 threads. starting thread 0 started thread 0 MotionThreadFunc thread started ven = NVIDIA Corporation ven = NVIDIA Corporation /home/pan/anaconda3/envs/dex/lib/python3.8/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32 warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow')) EGL device choice: -1 of 3. Loaded EGL 1.5 after reload. Failed to make context current (eglError: 12288)

[arduino183454]

Thank you very much for your patient reply. I will take a closer look at the code. Thank you for your help.