Adding Robot cause problem

[Nash-dc]

HI Mr.Hoang, I already integrate Multiverse in my thesis, thank you again! I found a very weird thing when I run demo multi_mujoco_sim_force I run the command roslaunch mujoco_sim multi_mujoco_sim_force.launch then everything looks perfect.When I move the cube in runtime,the cube in MuJoCo is also moved in the meantime Screencast from 17.08.2023 15:56:39.webm But after I add a robot(DualPanda for example) in scene and MultiverseActor.Then I move the cube in Unreal, however the cube in MuJoCo keep still,and the sphere didn't fall. Screencast from 17.08.2023 16:28:52.webm

All these weird things happen after Adding the Robot in MultiverseActor .I tried to find a reason in source code.But after go through the file mj_multiverse_client.cpp,I stll got no clue. It would be appreciated that you can give me some brief guidance!

[HoangGiang93]

I'm refactoring my framework structure. It's almost done, tomorrow I will notify if I checked everything ok

[Nash-dc]

thanks for your work! hope everything goes well

On Thu, Aug 17, 2023, 17:03 Giang Hoang Nguyen @.***> wrote:

I'm refactoring my framework structure. It's almost done, tomorrow I will notify if I checked everything ok

— Reply to this email directly, view it on GitHub https://github.com/Multiverse-Framework/Multiverse/issues/11#issuecomment-1682445191, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATRUBJGJVDY6F5D7WYMRVVTXVYXDHANCNFSM6AAAAAA3UFZDJU . You are receiving this because you authored the thread.Message ID: @.***>

[HoangGiang93]

I updated my code in Multiverse and Multiverse-UnrealEngine-Demos. You can follow my video to try it out. About your question, if you want to add a new robot in Unreal and controlled by MuJoCo, try the second one in the video. Look into the launch file I created to understand how to configure a new one. Sorry currently I'm busy to write a proper tutorial, it will happen in the next few weeks.

[Nash-dc]

Hi Mr.Hoang, Thanks for your reply and instructions. When I rebuild Multiverse,some problem occured,after I run ./build_multiverse.sh

[90%] Building multiverse_parser package
/bin/sh: 1: /home/dacheng/Documents/Multiverse/multiverse/bin/python3.10: not found
make[2]: *** [modules/multiverse_parser/CMakeFiles/multiverse_parser_build.dir/build.make:71: modules/multiverse_parser/CMakeFiles/multiverse_parser_build] Error 127
make[2]: Leaving directory '/home/dacheng/Documents/Multiverse/multiverse/build/multiverse'
make[1]: *** [CMakeFiles/Makefile2:292: modules/multiverse_parser/CMakeFiles/multiverse_parser_build.dir/all] Error 2
make[1]: Leaving directory '/home/dacheng/Documents/Multiverse/multiverse/build/multiverse'
make: *** [Makefile:136: all] Error 2
make: Leaving directory '/home/dacheng/Documents/Multiverse/multiverse/build/multiverse'
-- Install configuration: ""
-- Up-to-date: /home/dacheng/Documents/Multiverse/multiverse/bin/multiverse_server
-- Up-to-date: /home/dacheng/Documents/Multiverse/multiverse/lib/libstdc++/libmultiverse_client.a
-- Up-to-date: /home/dacheng/Documents/Multiverse/multiverse/lib/libstdc++/libmultiverse_client_json.so
-- Up-to-date: /home/dacheng/Documents/Multiverse/multiverse/lib/libstdc++/python/multiverse_client_pybind.cpython-38-x86_64-linux-gnu.so

I checked the python3.10,and I found it is broken It seems there are some problems in blender.So I checked blender file there is no bin file

But Multiverserver can run properly,and the connection to Unreal is established, so I just leave it.

Back to my question, I want to add a new robot in Unreal and controlled by MuJoCo, and add a cube in MuJoCo and controlled by Unreal, so I set the MultiverseActor like this:

and I modify the empty.xml and ur5e.yaml like this:

The robot connection runs properly,but cube connection is not established:

Screencast from 20.08.2023 10:31:47.webm Your guidance on setting up the cube connection would be greatly appreciated. Thank you again for your detailed video!

[HoangGiang93]

It couldn't move because of probably 2 reasons: 1) You set the constraint to ball. When you have a constraint, you can't freely move the object as you want in Unreal. Try changing to freejoint instead. 2) The joint and body name are the same. It could cause problem, try to change the joint name and see if it works.

About the installation, can you run sudo ./install.sh and ./build_third_parties.sh again? I guess you didn't do ./build_third_parties.sh

[HoangGiang93]

And another thing, you should modify the yaml file like this:

send:
    ur5e: [position, quaternion]
    joint: [joint_rvalue, joint_tvalue]

  receive:
    StaticMeshActor_1: [position, quaternion]

As before, if you include body: [position, quaternion] in send, it will consider all objects from MuJoCo is to be sent.

[Nash-dc]

Thanks for your reply! Actually I follow this instruction to build the Multiverse:

sudo ./install.sh      # Install all prerequisites
./build_third_parties.sh        # Build the dependencies
./build_multiverse.sh           # Build the software
./build_multiverse_ws.sh        # Build the ROS workspace

I just delete the Multiverse and build it again,the same problem still remains. I modified the yaml file like this as you said then I run roslaunch ur_mujoco mujoco.launch then I get the following errors:

dacheng@MyWondefulComputer:~$ roslaunch ur_mujoco mujoco.launch
... logging to /home/dacheng/.ros/log/6801fcae-3f43-11ee-a83c-4353074b01cf/roslaunch-MyWondefulComputer-11774.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

RLException: error loading <rosparam> tag: 
    file /home/dacheng/Documents/Multiverse/multiverse_ws/src/Multiverse-Robots/universal_robots/ur_mujoco/config/ur5e.yaml contains invalid YAML:
while parsing a block mapping
  in "<unicode string>", line 15, column 3:
      send:
      ^
expected <block end>, but found '<block mapping start>'
  in "<unicode string>", line 19, column 5:
        receive:
        ^
XML is <rosparam command="load" file="$(arg config)"/>
The traceback for the exception was written to the log file

It seems like this is not a correct form, so I modify like this: I also changed it to freejoint:

then it look like this: Screencast from 20.08.2023 12:34:29.webm By the way, if I remove the receive object in MultiverseActor and change the yaml file like this:

then the cube connection is established:

Screencast from 20.08.2023 12:39:01.webm

[Nash-dc]

I know what cause the installation issue.The blender requires GCC 11.0.0, but I have GCC 9.4.0.I solved it by updating my GCC version, thanks a lot!

[HoangGiang93]

The yaml file should be like this:

Can you send the log of the installation?

[HoangGiang93]

I know what cause the installation issue.The blender requires GCC 11.0.0, but I have GCC 9.4.0.I solved it by updating my GCC version, thanks a lot!

I'm aware of that, therefore I explicitly put GCC 11 in the install.sh and change its version. The fact that your installation still using GCC 9.4.0 confused me :/

[Nash-dc]

If I run sudo apt-get install gcc-11 I got the following errors

dacheng@MyWondefulComputer:~/Documents/Multiverse$ sudo apt-get install gcc-11
[sudo] password for dacheng: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package gcc-11

I think that is the reason why it didn't work

[Nash-dc]

It would be great if you can add this command in install.sh sudo add-apt-repository ppa:ubuntu-toolchain-r/test then the gcc-11 can be definitely found.

[Nash-dc]

Thank you Mr.Hoang! I changed the body to ur5e then the connection is established. I truly appreciate your consistent assistance. Please feel free to reach out if there's anything I can do to assist you in return.

[HoangGiang93]

Thank you Mr.Hoang! I changed the body to ur5e then the connection is established. I truly appreciate your consistent assistance. Please feel free to reach out if there's anything I can do to assist you in return.

I'm glad to hear that. Your interest on my framework is already a pleasure for me. And yes, I will add add-apt-repository ppa:ubuntu-toolchain-r/test to install.sh. If you found anything like that, please let me know. Creating an installation process that works globally is really not trivial at all ;__;