No GUI Appears when calling gym.make

[mphamhung]

When creating the environment in a python script (I'm running in a jupyter notebook),

env = gym.make('NoObstacleNavigationMir100Sim-v0', ip=target_machine_ip, gui=True)

No GUI pops up. But running from terminal roslaunch mir100_robot_server sim_robot_server.launch gui:=true

RVIZ/Gazebo pops up fine.

[matteolucchi]

Hi, when starting from the notebook does the environment successfully start? Do you get the print 'Successfully started Robot Server ...' like when you start it via terminal?

[mphamhung]

Yes, I do get it. I'm also able to do env.step() as well (which does show changes in the state each step).

[matteolucchi]

OK that is very weird indeed. You will need to access the tmux session to get access to some debug info. The simulation is run in the background by using tmux. Here you can find some useful tmux commands: cheatsheet.

After you started the environment in the notebook, in the same terminal where you started the server manager run tmux -L ServerManager this will connect you to the tmux server. Now you have to navigate to the session where the gazebo simulation has been started. You can move to the next session with Ctrl-B ).

Once you get to the session with the ROS output you have to use Ctrl-B + [ to be able to scroll through the output.

Please have a look in there and send me some screenshots of the output.

Once you are done you can detach from the session with Ctrl-B D

[mphamhung]

Thanks for the tmux instructions! It will be helpful in debugging the future.

There's an issue with connecting to the display:

QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-user' No protocol specified qt.qpa.screen: QXcbConnection: Could not connect to display :0 Could not connect to any X display. [rviz_asblab_18043_2315017222364237998-15] process has died [pid 18357, exit code 1, cmd /opt/ros/melodic/lib/rviz/rviz -d /home/user/robogym_ws/src/robo-gym-robot-servers/mir100_robot_server/rviz/mir_sim_rl.rviz __name:=rviz_asblab_18043_2315017222364237998 __log:=/home/user/.ros/log/70b28e2c-ae9f-11eb-a5d0-38d547adee32/rviz_asblab_18043_2315017222364237998-15.log]. log file: /home/user/.ros/log/70b28e2c-ae9f-11eb-a5d0-38d547adee32/rviz_asblab_18043_2315017222364237998-15*.log

I also get the same error when launching RVIZ from the same tmux session (After I CTRL+C), but not from a new one.

[mphamhung]

Upon checking the DISPLAY env variable, the session manager changes it to :0, whereas a fresh tmux session has it set to :1 (which works).

I can change the DISPLAY variable in the same session and launch the ros command again and it works fine (can still interface with the jupyter notebook) but not sure how to do that automatically.

[matteolucchi]

Ok thanks for the feedback, yes we are aware of this problem. The thing is that the X server is set differently for every user and we are looking at a solution on how to make this work for everyone. We set this variable here

I would suggest you install the robot-gym-server-modules from source with

pip install -e . and change the variable manually to 1