search

HoangGiang93 / URoboViz

Unreal plugin for robot visualisation using ROS connecting with WebSockets.
MIT License
67 stars 7 forks source link

Some problems I encountered during connecting UE4.27 and MuJoCo #3

Closed Nash-dc closed 1 year ago

Nash-dc commented 1 year ago

Dear Mr.Hoang,

This URoboViz is a really wonderful project, my final thesis was inspired by this project a lot, I would like to express my heartfelt thanks and compliment. But I have some problems during connecting UE4.27 and MuJoCo, I am a beginner to UE4 and ROS, it would be very appreciated that you can give me a biref explanation.

  1. In your project urdf_importer, in Quickstart.mp4 video: https://github.com/HoangGiang93/urdf_importer you typed: xacro /opt/ros/noetic/share/pr2_description/robots/pr2.urdf.xacro > pr2.urdf in the Terminal, but I checked this dictionary, the file pr2_description and its subdictionary does not exist. Where did you find this file? My ROS distro is as same as you(noetic).

  2. In this Connect with other physic enginehttps://github.com/HoangGiang93/URoboViz/wiki/4.-Connect-with-other-physics-engine in third section, you wrote: Launch the corresponding robot, in this example roslaunch tiago_mujoco tiago_display.launch Is this example stand for pr2.urdf which is mentioned before? But in Unreal Engine we use FBX format, hier is URDF format. How can these two connect each other?

As I said before I am not familiar with ROS and UE4. It would be really appreciated that you can help me get better understanding. Thanks in advance.

HoangGiang93 commented 1 year ago

Thank you for your nice comment, I'm glad to answer your issue:

  1. This is a standard ROS package fom ROS, therefore you can install this package from apt by command sudo apt install ros-noetic-pr2-description
  2. Unreal Engine use FBX format, but we have URDF format, that's why you need that urdf_importer to convert the robot from URDF format to FBX format. Please watch that video till the end so you can see the result. You could find more robots in URDF from my other packages https://github.com/HoangGiang93/mujoco_robots

Have fun!

Nash-dc commented 1 year ago

Thanks for your kindness and reply! Regarding the second question, I was wondering that the if the file _tiagodisplay.launch refers to the pr2 robot,cause you wrote Launch the corresponding robot , I want to make sure the corresponding robot is the pr2 robot.

HoangGiang93 commented 1 year ago

No it's not, in the example I showed is the tiago_robot. I didn't include any launch file for the pr2 robot. You can set it up and see how it goes. If you want to setup the tiago robot, all you need to do is to convert the URDF of tiago and convert to FBX using my blender add-on. And then import it in Unreal Engine and follow the tutorials.

Nash-dc commented 1 year ago

Everything understood !! Thanks for your reply and patience again!!! Hope you have a nice night

Nash-dc commented 1 year ago

Hi Mr.Hoang I would like to make sure that I convert the right corresponding robot for Tiago. First, I found the Tiago model in _/home/MyName/mujoco_ws/src/mujoco_robots/tiago/tiago_mujoco/model/tiago.urdf_ then I convert it in FBX format then I follow the tutorials, but when I launch roslaunch tiago_mujoco tiago_display.launch it seems there is no connection between UE4 and MuJoCo and a error: [tiago/rqt_robot_steering-5] process has died [pid 12520, exit code -6, cmd /opt/ros/noetic/lib/rqt_robot_steering/rqt_robot_steering __name:=rqt_robot_steering __log:=/home/MyName/.ros/log/fee0bf64-ea04-11ed-8b5b-2b88bc8805ec/tiago-rqt_robot_steering-5.log]. log file: /home/dacheng/.ros/log/fee0bf64-ea04-11ed-8b5b-2b88bc8805ec/tiago-rqt_robot_steering-5*.log so I want to make sure if I converted the correct tiago.urdf...

Thanks again for your patience

HoangGiang93 commented 1 year ago

If that is the only error your have then it seems like you don't have the package for the rqt_robot_steering node. You can install that from apt using this command in terminal sudo apt install ros-noetic-rqt-robot-steering.

Regardings the connection, if you can start the simulation and see it working, then the problem is only about the connection. For this you need to launch a rosbridge server. You can install that from apt using this command in terminal sudo apt install ros-noetic-rosbridge-suite.

After installing the rosbridge_server package, run this launch file before starting Unreal simulation:

roslaunch rosbridge_server rosbridge_websocket.launch port:=9393

Then you could start the Unreal simulation and it will visualize what MuJoCo does. Keep in mind that after starting the rosbridge, you can start and stop Unreal simulation whenever you want.

If the connection is still not established, perhaps you could run rostopic list first before starting Unreal simulation to check if all the ROS topic names are correctly written in ROSManager from your RoboManager Actor in the scene. If the ROS topic names are correct and it still doesn't move, then try to run rostopic echo <topic_name> to see if any signals come from MuJoCo, like rostopic echo /<namespace>/joint_states. If the signals are published but the robot still doesn't move, try to command rostopic info <topic_name> to check if there are subscribers from Unreal simulation via rosbridge server. If all the connections are correctly defined and the message are there but the robot still not moves, try to restart everything.

Hope that I covered every scenarios that could happen ;)

Nash-dc commented 1 year ago

Thanks for your detailed explanation! You really help me a lot, please let me buy a coffee for you...

After I run sudo apt install ros-noetic-rqt-robot-steering,it shows I already installed the package:

Reading package lists... Done
Building dependency tree       
Reading state information... Done
ros-noetic-rqt-robot-steering is already the newest version (0.5.12-1focal.20230215.222145).
0 upgraded, 0 newly installed, 0 to remove and 21 not upgraded.

and this error still exist:

[tiago/rqt_robot_steering-5] process has died [pid 12520, exit code -6, cmd /opt/ros/noetic/lib/rqt_robot_steering/rqt_robot_steering __name:=rqt_robot_steering __log:=/home/MyName/.ros/log/fee0bf64-ea04-11ed-8b5b-2b88bc8805ec/tiago-rqt_robot_steering-5.log]. log file: /home/MyName/.ros/log/fee0bf64-ea04-11ed-8b5b-2b88bc8805ec/tiago-rqt_robot_steering-5*.log

After I installed rosbridge_server package, the connection is still not established, and I run rostopic list, the ouput is:

rosout
/rosout_agg

The ROS topic names are not correctly written in ROSManager, though I added the correct ROS subscriber. Thanks for you help anyway, please give a chance to buy you a cup of coffee.

HoangGiang93 commented 1 year ago

that's weird :/ it looks like nothing is running on your computer. Did the simulation run? Could you see something pop up? It might be better if you can post me a screenshot

Nash-dc commented 1 year ago

1.webm

Nash-dc commented 1 year ago

2.webm

Nash-dc commented 1 year ago

3.webm

Nash-dc commented 1 year ago

4.webm

HoangGiang93 commented 1 year ago

You have a very weird computer ;__; normally it should show the topic from rosbridge

  1. After you run mujoco, can you run rostopic list in a new terminal again? Also maybe try echo $ROS_MASTER_URI
  2. Can you enlarge the terminal more so I could see all the log and errors it show? Or maybe copy down in the chat
Nash-dc commented 1 year ago

Here is the video: 5.zip ROSbridge log:

dacheng@dacheng-Legion-5-Pro-16ACH6H:~$ roslaunch rosbridge_server rosbridge_websocket.launch port:=9393
... logging to /home/dacheng/.ros/log/b6771c68-eabe-11ed-874f-81090c3b8986/roslaunch-dacheng-Legion-5-Pro-16ACH6H-7152.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://dacheng-Legion-5-Pro-16ACH6H:44813/

SUMMARY
========

PARAMETERS
 * /rosapi/params_glob: [*]
 * /rosapi/services_glob: [*]
 * /rosapi/topics_glob: [*]
 * /rosbridge_websocket/address: 0.0.0.0
 * /rosbridge_websocket/authenticate: False
 * /rosbridge_websocket/bson_only_mode: False
 * /rosbridge_websocket/delay_between_messages: 0
 * /rosbridge_websocket/fragment_timeout: 600
 * /rosbridge_websocket/max_message_size: None
 * /rosbridge_websocket/params_glob: [*]
 * /rosbridge_websocket/port: 9393
 * /rosbridge_websocket/retry_startup_delay: 5
 * /rosbridge_websocket/services_glob: [*]
 * /rosbridge_websocket/topics_glob: [*]
 * /rosbridge_websocket/unregister_timeout: 10
 * /rosbridge_websocket/use_compression: False
 * /rosbridge_websocket/websocket_external_port: None
 * /rosbridge_websocket/websocket_ping_interval: 0
 * /rosbridge_websocket/websocket_ping_timeout: 30
 * /rosdistro: noetic
 * /rosversion: 1.16.0

NODES
  /
    rosapi (rosapi/rosapi_node)
    rosbridge_websocket (rosbridge_server/rosbridge_websocket)

ROS_MASTER_URI=http://localhost:11311

process[rosbridge_websocket-1]: started with pid [7196]
process[rosapi-2]: started with pid [7197]
:0: UserWarning: You do not have a working installation of the service_identity module: 'No module named 'service_identity''.  Please install it from <https://pypi.python.org/pypi/service_identity> and make sure all of its dependencies are satisfied.  Without the service_identity module, Twisted can perform only rudimentary TLS client hostname verification.  Many valid certificate/hostname mappings may be rejected.
/opt/ros/noetic/lib/rosbridge_server/rosbridge_websocket:41: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
  from distutils.version import LooseVersion
2023-05-04 23:06:27+0200 [-] Log opened.
2023-05-04 23:06:28+0200 [-] registered capabilities (classes):
2023-05-04 23:06:28+0200 [-]  - <class 'rosbridge_library.capabilities.call_service.CallService'>
2023-05-04 23:06:28+0200 [-]  - <class 'rosbridge_library.capabilities.advertise.Advertise'>
2023-05-04 23:06:28+0200 [-]  - <class 'rosbridge_library.capabilities.publish.Publish'>
2023-05-04 23:06:28+0200 [-]  - <class 'rosbridge_library.capabilities.subscribe.Subscribe'>
2023-05-04 23:06:28+0200 [-]  - <class 'rosbridge_library.capabilities.defragmentation.Defragment'>
2023-05-04 23:06:28+0200 [-]  - <class 'rosbridge_library.capabilities.advertise_service.AdvertiseService'>
2023-05-04 23:06:28+0200 [-]  - <class 'rosbridge_library.capabilities.service_response.ServiceResponse'>
2023-05-04 23:06:28+0200 [-]  - <class 'rosbridge_library.capabilities.unadvertise_service.UnadvertiseService'>

UnrealEngine log:

LogWorld: BeginTearingDown for /Game/StarterContent/Maps/UEDPIE_0_Minimal_Default
LogWorld: UWorld::CleanupWorld for Minimal_Default, bSessionEnded=true, bCleanupResources=true
LogSlate: InvalidateAllWidgets triggered.  All widgets were invalidated
LogPlayLevel: Display: Shutting down PIE online subsystems
LogSlate: InvalidateAllWidgets triggered.  All widgets were invalidated
LogAudio: Display: Audio Device unregistered from world 'None'.
LogSlate: Updating window title bar state: overlay mode, drag disabled, window buttons hidden, title bar hidden
LogUObjectHash: Compacting FUObjectHashTables data took   0.29ms
LogPlayLevel: Display: Destroying online subsystem :Context_2
LogDebuggerCommands: Repeating last play command: Selected Viewport
LogPlayLevel: PlayLevel: No blueprints needed recompiling
PIE: New page: PIE session: Minimal_Default (May 5, 2023, 1:04:29 AM)
LogPlayLevel: Creating play world package: /Game/StarterContent/Maps/UEDPIE_0_Minimal_Default
LogPlayLevel: PIE: StaticDuplicateObject took: (0.004082s)
LogPlayLevel: PIE: Created PIE world by copying editor world from /Game/StarterContent/Maps/Minimal_Default.Minimal_Default to /Game/StarterContent/Maps/UEDPIE_0_Minimal_Default.Minimal_Default (0.004149s)
LogSubsystemCollection: Failed to initialize subsystem dependency (WorldPartitionSubsystem)
LogChaos: FPhysicsSolverBase::AsyncDt:-1.000000
LogAIModule: Creating AISystem for world Minimal_Default
LogPlayLevel: PIE: World Init took: (0.000758s)
LogUObjectHash: Compacting FUObjectHashTables data took   0.37ms
LogAudio: Display: Creating Audio Device:                 Id: 3, Scope: Unique, Realtime: True
LogAudioMixer: Display: Audio Mixer Platform Settings:
LogAudioMixer: Display:     Sample Rate:                          48000
LogAudioMixer: Display:     Callback Buffer Frame Size Requested: 1024
LogAudioMixer: Display:     Callback Buffer Frame Size To Use:    1024
LogAudioMixer: Display:     Number of buffers to queue:           2
LogAudioMixer: Display:     Max Channels (voices):                0
LogAudioMixer: Display:     Number of Async Source Workers:       0
LogAudio: Display: AudioDevice MaxSources: 32
LogAudio: Display: Audio Spatialization Plugin: None (built-in).
LogAudio: Display: Audio Reverb Plugin: None (built-in).
LogAudio: Display: Audio Occlusion Plugin: None (built-in).
LogAudioMixerSDL: Display: Initialized SDL using pulseaudio platform API backend.
LogAudioMixer: Display: Initializing audio mixer using platform API: 'SDL2'
LogAudioMixerSDL: Opening default audio device (device index -1)
LogAudioMixer: Display: Using Audio Hardware Device Default Audio Device
LogAudioMixer: Display: Initializing Sound Submixes...
LogAudioMixer: Display: Creating Master Submix 'MasterSubmixDefault'
LogAudioMixer: Display: Creating Master Submix 'MasterReverbSubmixDefault'
LogAudioMixer: Display: Output buffers initialized: Frames=1024, Channels=6, Samples=6144, InstanceID=3
LogAudioMixer: Display: Starting AudioMixerPlatformInterface::RunInternal(), InstanceID=3
LogInit: FAudioDevice initialized.
LogAudio: Display: Audio Device (ID: 3) registered with world 'Minimal_Default'.
LogLoad: Game class is 'GameModeBase'
LogWorld: Bringing World /Game/StarterContent/Maps/UEDPIE_0_Minimal_Default.Minimal_Default up for play (max tick rate 0) at 2023.05.04-23.04.29
LogWorld: Bringing up level for play took: 0.001339
LogOnline: OSS: Created online subsystem instance for: :Context_3
LogROS: Error: [OnError] Error in Websocket.
PIE: Server logged in
PIE: Play in editor total start time 0.099 seconds.
LogROS: Error: [OnError] Error in Websocket.
LogROS: Error: [Run] Could not connect to the rosbridge server (IP 127.0.0.1, port 9393)!
LogROS: Error: [Run] Could not connect to the rosbridge server (IP 127.0.0.1, port 9393)!
LogSlate: Updating window title bar state: overlay mode, drag disabled, window buttons hidden, title bar hidden
LogWorld: BeginTearingDown for /Game/StarterContent/Maps/UEDPIE_0_Minimal_Default
LogWorld: UWorld::CleanupWorld for Minimal_Default, bSessionEnded=true, bCleanupResources=true
LogSlate: InvalidateAllWidgets triggered.  All widgets were invalidated
LogPlayLevel: Display: Shutting down PIE online subsystems
LogSlate: InvalidateAllWidgets triggered.  All widgets were invalidated
LogSlate: Updating window title bar state: overlay mode, drag disabled, window buttons hidden, title bar hidden
LogUObjectHash: Compacting FUObjectHashTables data took   0.34ms
LogPlayLevel: Display: Destroying online subsystem :Context_3
LogDebuggerCommands: Repeating last play command: Selected Viewport
LogPlayLevel: PlayLevel: No blueprints needed recompiling
PIE: New page: PIE session: Minimal_Default (May 5, 2023, 1:06:35 AM)
LogPlayLevel: Creating play world package: /Game/StarterContent/Maps/UEDPIE_0_Minimal_Default
LogPlayLevel: PIE: StaticDuplicateObject took: (0.003730s)
LogPlayLevel: PIE: Created PIE world by copying editor world from /Game/StarterContent/Maps/Minimal_Default.Minimal_Default to /Game/StarterContent/Maps/UEDPIE_0_Minimal_Default.Minimal_Default (0.003792s)
LogSubsystemCollection: Failed to initialize subsystem dependency (WorldPartitionSubsystem)
LogChaos: FPhysicsSolverBase::AsyncDt:-1.000000
LogAIModule: Creating AISystem for world Minimal_Default
LogPlayLevel: PIE: World Init took: (0.000808s)
LogUObjectHash: Compacting FUObjectHashTables data took   0.38ms
LogAudio: Display: Creating Audio Device:                 Id: 4, Scope: Unique, Realtime: True
LogAudioMixer: Display: Audio Mixer Platform Settings:
LogAudioMixer: Display:     Sample Rate:                          48000
LogAudioMixer: Display:     Callback Buffer Frame Size Requested: 1024
LogAudioMixer: Display:     Callback Buffer Frame Size To Use:    1024
LogAudioMixer: Display:     Number of buffers to queue:           2
LogAudioMixer: Display:     Max Channels (voices):                0
LogAudioMixer: Display:     Number of Async Source Workers:       0
LogAudio: Display: AudioDevice MaxSources: 32
LogAudio: Display: Audio Spatialization Plugin: None (built-in).
LogAudio: Display: Audio Reverb Plugin: None (built-in).
LogAudio: Display: Audio Occlusion Plugin: None (built-in).
LogAudioMixerSDL: Display: Initialized SDL using pulseaudio platform API backend.
LogAudioMixer: Display: Initializing audio mixer using platform API: 'SDL2'
LogAudioMixerSDL: Opening default audio device (device index -1)
LogAudioMixer: Display: Using Audio Hardware Device Default Audio Device
LogAudioMixer: Display: Initializing Sound Submixes...
LogAudioMixer: Display: Creating Master Submix 'MasterSubmixDefault'
LogAudioMixer: Display: Creating Master Submix 'MasterReverbSubmixDefault'
LogAudioMixer: Display: Output buffers initialized: Frames=1024, Channels=6, Samples=6144, InstanceID=4
LogAudioMixer: Display: Starting AudioMixerPlatformInterface::RunInternal(), InstanceID=4
LogInit: FAudioDevice initialized.
LogAudio: Display: Audio Device (ID: 4) registered with world 'Minimal_Default'.
LogLoad: Game class is 'GameModeBase'
LogWorld: Bringing World /Game/StarterContent/Maps/UEDPIE_0_Minimal_Default.Minimal_Default up for play (max tick rate 0) at 2023.05.04-23.06.35
LogWorld: Bringing up level for play took: 0.001266
LogOnline: OSS: Created online subsystem instance for: :Context_4
PIE: Server logged in
PIE: Play in editor total start time 0.106 seconds.
LogROS: Error: [OnError] Error in Websocket.
LogROS: Error: [OnError] Error in Websocket.
LogROS: Error: [Run] Could not connect to the rosbridge server (IP 127.0.0.1, port 9393)!
LogROS: Error: [Run] Could not connect to the rosbridge server (IP 127.0.0.1, port 9393)!
LogViewport: Scene viewport resized to 1152x1204, mode Windowed.

rostopic list log(after run roslaunch tiago_mujoco tiago_display.launch):

dacheng@dacheng-Legion-5-Pro-16ACH6H:~$ rostopic list
/clicked_point
/initialpose
/move_base_simple/goal
/mujoco/object_joint_states
/mujoco/object_states
/mujoco/robot_joint_states
/mujoco/sensors_3D
/mujoco/visualization_marker_array
/mujoco/world_joint_states
/rosout
/rosout_agg
/tf
/tf_static
/tiago/base_footprint
/tiago/cmd_vel
/tiago/joint_states

after run roslaunch tiago_mujoco tiago_display.launch:

... logging to /home/dacheng/.ros/log/b6771c68-eabe-11ed-874f-81090c3b8986/roslaunch-dacheng-Legion-5-Pro-16ACH6H-7293.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://dacheng-Legion-5-Pro-16ACH6H:39887/

SUMMARY
========

PARAMETERS
 * /rosdistro: noetic
 * /rosversion: 1.16.0
 * /tiago/arm_left_controller/constraints/arm_left_1_joint/goal: 0.02
 * /tiago/arm_left_controller/constraints/arm_left_2_joint/goal: 0.02
 * /tiago/arm_left_controller/constraints/arm_left_3_joint/goal: 0.02
 * /tiago/arm_left_controller/constraints/arm_left_4_joint/goal: 0.02
 * /tiago/arm_left_controller/constraints/arm_left_5_joint/goal: 0.02
 * /tiago/arm_left_controller/constraints/arm_left_6_joint/goal: 0.02
 * /tiago/arm_left_controller/constraints/arm_left_7_joint/goal: 0.02
 * /tiago/arm_left_controller/constraints/goal_time: 0.6
 * /tiago/arm_left_controller/constraints/stopped_velocity_tolerance: 5.0
 * /tiago/arm_left_controller/gains/arm_left_1_joint/d: 50
 * /tiago/arm_left_controller/gains/arm_left_1_joint/i: 100
 * /tiago/arm_left_controller/gains/arm_left_1_joint/p: 2000
 * /tiago/arm_left_controller/gains/arm_left_2_joint/d: 50
 * /tiago/arm_left_controller/gains/arm_left_2_joint/i: 100
 * /tiago/arm_left_controller/gains/arm_left_2_joint/p: 2000
 * /tiago/arm_left_controller/gains/arm_left_3_joint/d: 50
 * /tiago/arm_left_controller/gains/arm_left_3_joint/i: 100
 * /tiago/arm_left_controller/gains/arm_left_3_joint/p: 2000
 * /tiago/arm_left_controller/gains/arm_left_4_joint/d: 50
 * /tiago/arm_left_controller/gains/arm_left_4_joint/i: 100
 * /tiago/arm_left_controller/gains/arm_left_4_joint/p: 2000
 * /tiago/arm_left_controller/gains/arm_left_5_joint/d: 50
 * /tiago/arm_left_controller/gains/arm_left_5_joint/i: 100
 * /tiago/arm_left_controller/gains/arm_left_5_joint/p: 2000
 * /tiago/arm_left_controller/gains/arm_left_6_joint/d: 50
 * /tiago/arm_left_controller/gains/arm_left_6_joint/i: 100
 * /tiago/arm_left_controller/gains/arm_left_6_joint/p: 2000
 * /tiago/arm_left_controller/gains/arm_left_7_joint/d: 50
 * /tiago/arm_left_controller/gains/arm_left_7_joint/i: 100
 * /tiago/arm_left_controller/gains/arm_left_7_joint/p: 2000
 * /tiago/arm_left_controller/joints: ['arm_left_1_join...
 * /tiago/arm_left_controller/type: effort_controller...
 * /tiago/arm_right_controller/constraints/arm_right_1_joint/goal: 0.02
 * /tiago/arm_right_controller/constraints/arm_right_2_joint/goal: 0.02
 * /tiago/arm_right_controller/constraints/arm_right_3_joint/goal: 0.02
 * /tiago/arm_right_controller/constraints/arm_right_4_joint/goal: 0.02
 * /tiago/arm_right_controller/constraints/arm_right_5_joint/goal: 0.02
 * /tiago/arm_right_controller/constraints/arm_right_6_joint/goal: 0.02
 * /tiago/arm_right_controller/constraints/arm_right_7_joint/goal: 0.02
 * /tiago/arm_right_controller/constraints/goal_time: 0.6
 * /tiago/arm_right_controller/constraints/stopped_velocity_tolerance: 5.0
 * /tiago/arm_right_controller/gains/arm_right_1_joint/d: 50
 * /tiago/arm_right_controller/gains/arm_right_1_joint/i: 100
 * /tiago/arm_right_controller/gains/arm_right_1_joint/p: 2000
 * /tiago/arm_right_controller/gains/arm_right_2_joint/d: 50
 * /tiago/arm_right_controller/gains/arm_right_2_joint/i: 100
 * /tiago/arm_right_controller/gains/arm_right_2_joint/p: 2000
 * /tiago/arm_right_controller/gains/arm_right_3_joint/d: 50
 * /tiago/arm_right_controller/gains/arm_right_3_joint/i: 100
 * /tiago/arm_right_controller/gains/arm_right_3_joint/p: 2000
 * /tiago/arm_right_controller/gains/arm_right_4_joint/d: 50
 * /tiago/arm_right_controller/gains/arm_right_4_joint/i: 100
 * /tiago/arm_right_controller/gains/arm_right_4_joint/p: 2000
 * /tiago/arm_right_controller/gains/arm_right_5_joint/d: 50
 * /tiago/arm_right_controller/gains/arm_right_5_joint/i: 100
 * /tiago/arm_right_controller/gains/arm_right_5_joint/p: 2000
 * /tiago/arm_right_controller/gains/arm_right_6_joint/d: 50
 * /tiago/arm_right_controller/gains/arm_right_6_joint/i: 100
 * /tiago/arm_right_controller/gains/arm_right_6_joint/p: 2000
 * /tiago/arm_right_controller/gains/arm_right_7_joint/d: 50
 * /tiago/arm_right_controller/gains/arm_right_7_joint/i: 100
 * /tiago/arm_right_controller/gains/arm_right_7_joint/p: 2000
 * /tiago/arm_right_controller/joints: ['arm_right_1_joi...
 * /tiago/arm_right_controller/type: effort_controller...
 * /tiago/gripper_left_left_finger/gains/gripper_left_left_finger_joint/d: 50
 * /tiago/gripper_left_left_finger/gains/gripper_left_left_finger_joint/i: 100
 * /tiago/gripper_left_left_finger/gains/gripper_left_left_finger_joint/p: 2000
 * /tiago/gripper_left_left_finger/joint: gripper_left_left...
 * /tiago/gripper_left_left_finger/type: effort_controller...
 * /tiago/gripper_left_right_finger/gains/gripper_left_right_finger_joint/d: 50
 * /tiago/gripper_left_right_finger/gains/gripper_left_right_finger_joint/i: 100
 * /tiago/gripper_left_right_finger/gains/gripper_left_right_finger_joint/p: 2000
 * /tiago/gripper_left_right_finger/joint: gripper_left_righ...
 * /tiago/gripper_left_right_finger/type: effort_controller...
 * /tiago/gripper_right_left_finger/gains/gripper_right_left_finger_joint/d: 50
 * /tiago/gripper_right_left_finger/gains/gripper_right_left_finger_joint/i: 100
 * /tiago/gripper_right_left_finger/gains/gripper_right_left_finger_joint/p: 2000
 * /tiago/gripper_right_left_finger/joint: gripper_right_lef...
 * /tiago/gripper_right_left_finger/type: effort_controller...
 * /tiago/gripper_right_right_finger/gains/gripper_right_right_finger_joint/d: 50
 * /tiago/gripper_right_right_finger/gains/gripper_right_right_finger_joint/i: 100
 * /tiago/gripper_right_right_finger/gains/gripper_right_right_finger_joint/p: 2000
 * /tiago/gripper_right_right_finger/joint: gripper_right_rig...
 * /tiago/gripper_right_right_finger/type: effort_controller...
 * /tiago/head_controller/constraints/goal_time: 2.0
 * /tiago/head_controller/constraints/head_1_joint/goal: 0.02
 * /tiago/head_controller/constraints/head_2_joint/goal: 0.02
 * /tiago/head_controller/constraints/stopped_velocity_tolerance: 5.0
 * /tiago/head_controller/gains/head_1_joint/d: 50
 * /tiago/head_controller/gains/head_1_joint/i: 100
 * /tiago/head_controller/gains/head_1_joint/p: 2000
 * /tiago/head_controller/gains/head_2_joint/d: 50
 * /tiago/head_controller/gains/head_2_joint/i: 100
 * /tiago/head_controller/gains/head_2_joint/p: 2000
 * /tiago/head_controller/joints: ['head_1_joint', ...
 * /tiago/head_controller/type: effort_controller...
 * /tiago/joint_state_controller/publish_rate: 60
 * /tiago/joint_state_controller/type: joint_state_contr...
 * /tiago/mujoco_sim/add_odom_joints: True
 * /tiago/mujoco_sim/pub_base_pose_rate: 60.0
 * /tiago/mujoco_sim/pub_joint_states/world_bodies_rate: 60.0
 * /tiago/mujoco_sim/pub_object_state_array/world_bodies_rate: 0.0
 * /tiago/mujoco_sim/pub_tf/world_bodies_rate: 0.0
 * /tiago/mujoco_sim/robot: /home/dacheng/muj...
 * /tiago/mujoco_sim/root_frame_id: map
 * /tiago/mujoco_sim/spawn_and_destroy_objects_rate: 0.0
 * /tiago/mujoco_sim/world: /home/dacheng/muj...
 * /tiago/robot_description: <?xml version="1....
 * /tiago/torso_controller/constraints/goal_time: 0.6
 * /tiago/torso_controller/constraints/stopped_velocity_tolerance: 5.0
 * /tiago/torso_controller/constraints/torso_lift_joint/goal: 0.001
 * /tiago/torso_controller/gains/torso_lift_joint/d: 50
 * /tiago/torso_controller/gains/torso_lift_joint/i: 100
 * /tiago/torso_controller/gains/torso_lift_joint/p: 2000
 * /tiago/torso_controller/joints: ['torso_lift_joint']
 * /tiago/torso_controller/type: effort_controller...

NODES
  /tiago/
    controller_manager (controller_manager/controller_manager)
    mujoco_sim (mujoco_sim/mujoco_sim_node)
    robot_state_publisher (robot_state_publisher/robot_state_publisher)
    rqt_robot_steering (rqt_robot_steering/rqt_robot_steering)
    rviz (rviz/rviz)

ROS_MASTER_URI=http://localhost:11311

process[tiago/mujoco_sim-1]: started with pid [7354]
process[tiago/robot_state_publisher-2]: started with pid [7355]
process[tiago/controller_manager-3]: started with pid [7356]
process[tiago/rqt_robot_steering-4]: started with pid [7357]
process[tiago/rviz-5]: started with pid [7358]
[ INFO] [1683234400.407338594]: Get ROS parameters from server
[ INFO] [1683234400.408582291]: Set disable_gravity to true
[ INFO] [1683234400.409064767]: Set world from /home/dacheng/mujoco_ws/src/mujoco_robots/tiago/tiago_mujoco/model/world.xml
[ WARN] [1683234400.409353848]: Robot names not found in parameter server, searching for robot names from urdf...
[ INFO] [1683234400.418540385]: Found 1 robots: [tiago]
[ INFO] [1683234400.419780729]: Initializing MuJoCo simulator...
[ INFO] [1683234400.419807060]: Copying model in /home/dacheng/mujoco_ws/src/mujoco_sim/model/tmp...
[ INFO] [1683234400.420644968]: Adding odom joints for model tiago...
[ INFO] [1683234400.420675978]: Add 3 odom joints for model tiago successfully
[ INFO] [1683234400.420946830]: Save models in /home/dacheng/mujoco_ws/src/mujoco_sim/model/tmp successfully
[ INFO] [1683234400.444739242]: Reload model in /home/dacheng/mujoco_ws/src/mujoco_robots/tiago/tiago_mujoco/model/tiago.xml complete
[ INFO] [1683234400.445562622]: Getting joints of model tiago...
[ INFO] [1683234400.446033365]: Initialize model tiago with 33 joints successfully
[ INFO] [1683234400.446345424]: Initialize MuJoCo simulator successfully
[ INFO] [1683234400.446400251]: Initializing ROS interface...
[ INFO] [1683234400.463202413]: Started [/mujoco/screenshot] service.
[ INFO] [1683234400.463580474]: Started [/mujoco/reset] service.
[ INFO] [1683234400.463986751]: Started [/mujoco/spawn_objects] service.
[ INFO] [1683234400.464395543]: Started [/mujoco/destroy_objects] service.
[ WARN] [1683234400.464620298]: joint_inits not found, will set to default value (0)
[ INFO] [1683234400.466926588]: Initialize ROS interface successfully
[ INFO] [1683234400.466949497]: Initializing OpenGL...
[ INFO] [1683234400.668165671]: Initialize OpenGL successfully...
PluginManager._discover() force discovery of plugins
RospkgPluginProvider._find_plugins() crawling for plugins of type 'qt_gui'
ValueError: bad marshal data (unknown type code)
Fatal Python error: could not initialize part 1
Python runtime state: initialized

Current thread 0x00007f7aac482280 (most recent call first):
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1229 in create_module
  File "<frozen importlib._bootstrap>", line 573 in module_from_spec
  File "<frozen importlib._bootstrap>", line 676 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1149 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1178 in _find_and_load
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1234 in _handle_fromlist
  File "/opt/ros/noetic/lib/python3/dist-packages/qt_gui_cpp/cpp_binding_helper.py", line 39 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 936 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1149 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1178 in _find_and_load
  File "/opt/ros/noetic/lib/python3/dist-packages/qt_gui_cpp/cpp_plugin_provider.py", line 33 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 936 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1149 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1178 in _find_and_load
  File "/opt/ros/noetic/lib/python3/dist-packages/rqt_gui/ros_plugin_provider.py", line 79 in load
  File "/opt/ros/noetic/lib/python3/dist-packages/qt_gui/recursive_plugin_provider.py", line 60 in discover
  File "/opt/ros/noetic/lib/python3/dist-packages/qt_gui/composite_plugin_provider.py", line 57 in discover
  File "/opt/ros/noetic/lib/python3/dist-packages/qt_gui/plugin_manager.py", line 153 in _discover
  File "/opt/ros/noetic/lib/python3/dist-packages/qt_gui/plugin_manager.py", line 120 in discover
  File "/opt/ros/noetic/lib/python3/dist-packages/qt_gui/plugin_manager.py", line 170 in get_plugins
  File "/opt/ros/noetic/lib/python3/dist-packages/qt_gui/plugin_manager.py", line 163 in find_plugins_by_name
  File "/opt/ros/noetic/lib/python3/dist-packages/qt_gui/main.py", line 587 in main
  File "/opt/ros/noetic/lib/python3/dist-packages/rqt_gui/main.py", line 62 in main
  File "/opt/ros/noetic/lib/rqt_robot_steering/rqt_robot_steering", line 8 in <module>

Extension modules: yaml._yaml, xxsubtype, shiboken2.shiboken2, PySide2.QtCore, QtCore, PySide2.QtGui, QtGui, PySide2.QtWidgets, QtWidgets, PySide2.QtHelp, QtHelp, PySide2.QtPositioning, PySide2.QtLocation, QtLocation, PySide2.QtNetwork, PySide2.QtMultimedia, QtMultimedia, PySide2.QtMultimediaWidgets, QtMultimediaWidgets, QtNetwork, PySide2.QtOpenGL, QtOpenGL, QtPositioning, PySide2.QtPrintSupport, QtPrintSupport, PySide2.QtQml, QtQml, PySide2.QtQuick, QtQuick, PySide2.QtQuickWidgets, QtQuickWidgets, PySide2.QtScript, QtScript, PySide2.QtScriptTools, QtScriptTools, PySide2.QtSensors, QtSensors, PySide2.QtSql, QtSql, PySide2.QtSvg, QtSvg, PySide2.QtTest, QtTest, PySide2.QtWebChannel, QtWebChannel, PySide2.QtWebEngine, QtWebEngine, PySide2.QtWebEngineCore, QtWebEngineCore, PySide2.QtWebEngineWidgets, QtWebEngineWidgets, PySide2.QtWebSockets, QtWebSockets, PySide2.QtX11Extras, QtX11Extras, PySide2.QtXml, QtXml, PySide2.QtXmlPatterns, QtXmlPatterns, python_qt_binding.QtCore, python_qt_binding.QtGui, python_qt_binding.QtWidgets, python_qt_binding.QtHelp, python_qt_binding.QtLocation, python_qt_binding.QtMultimedia, python_qt_binding.QtMultimediaWidgets, python_qt_binding.QtNetwork, python_qt_binding.QtOpenGL, python_qt_binding.QtPositioning, python_qt_binding.QtPrintSupport, python_qt_binding.QtQml, python_qt_binding.QtQuick, python_qt_binding.QtQuickWidgets, python_qt_binding.QtScript, python_qt_binding.QtScriptTools, python_qt_binding.QtSensors, python_qt_binding.QtSql, python_qt_binding.QtSvg, python_qt_binding.QtTest, python_qt_binding.QtWebChannel, python_qt_binding.QtWebEngine, python_qt_binding.QtWebEngineCore, python_qt_binding.QtWebEngineWidgets, python_qt_binding.QtWebSockets, python_qt_binding.QtX11Extras, python_qt_binding.QtXml, python_qt_binding.QtXmlPatterns (total: 87)
[tiago/rqt_robot_steering-4] process has died [pid 7357, exit code -6, cmd /opt/ros/noetic/lib/rqt_robot_steering/rqt_robot_steering __name:=rqt_robot_steering __log:=/home/dacheng/.ros/log/b6771c68-eabe-11ed-874f-81090c3b8986/tiago-rqt_robot_steering-4.log].
log file: /home/dacheng/.ros/log/b6771c68-eabe-11ed-874f-81090c3b8986/tiago-rqt_robot_steering-4*.log
[tiago/mujoco_sim-1] process has finished cleanly
log file: /home/dacheng/.ros/log/b6771c68-eabe-11ed-874f-81090c3b8986/tiago-mujoco_sim-1*.log
[tiago/rviz-5] process has finished cleanly
log file: /home/dacheng/.ros/log/b6771c68-eabe-11ed-874f-81090c3b8986/tiago-rviz-5*.log

after run echo $ROS_MASTER_URI:

dacheng@dacheng-Legion-5-Pro-16ACH6H:~$ echo $ROS_MASTER_URI
http://localhost:11311

Many thanks!!!

HoangGiang93 commented 1 year ago

Ok I think your rosbridge has some problem, it doesn't run properly. Maybe try to remove and install it again?

If that is not working, try to remove it and use the package from source instead: https://github.com/RobotWebTools/rosbridge_suite

Just clone this packge inside the src folder in your workspace and build it again, then source it and launch it again.

Hope that work!

Nash-dc commented 1 year ago

Thanks again for your help. I uninstall the rosbridge and built it from source, then it works. Now the rosbridge is connected, but when I run rostopic echo joint_states,there is no response come from MuJoCo ,and it still doesn't move. 1.webm

check the video. 2.webm

HoangGiang93 commented 1 year ago

well it doesn't work like that, you have to manually drag the robot from Content into the viewport in Unreal Editor and then manually configure the correct parameter. You can revise again on https://github.com/HoangGiang93/URoboViz/wiki/2.-Setup-a-robot#3-use-the-robot-anim and https://github.com/HoangGiang93/URoboViz/wiki/3.-Control-a-robot-kinematically

Nash-dc commented 1 year ago

Thank you Hoang!!!You are very nice.

HoangGiang93 commented 1 year ago

You're welcome :)