microsoft / mixed-reality-robot-interaction-demo

Sample Unity application for Android and Hololens, showing how these devices can be used to interact with a (simulated or real) robot through Mixed Reality. Released as part of the IROS 2020 Tutorial on Mixed Reality and Robotics.
https://www.microsoft.com/en-us/research/event/mixed-reality-and-robotics-tutorial-iros-2020/
MIT License
56 stars 15 forks source link

Robot wheels are not responding #6

Closed PattyUnity closed 2 years ago

PattyUnity commented 3 years ago

Dear Eric,

Thank you for the demo! I am able to deploy the Hololens app and create ASA anchor successfully. However, I couldn’t send the robot to the waypoint marker. It seems the wheels are not responding.

From the PowerShell, there’s error saying "No p gain specified for pid. Namespace: /gazebo_ros_control/pid_gains/front_left_wheel".

May I have your guidance on this? Your help is highly appreciated.

Best Regards, Patt

EricVoll commented 3 years ago

Hi Patt, thank you for creating this issue and shining some light on it!

The issue is, that you PC seems to have an issue with launching Gazebo quickly enough, such that other nodes are timing out and then the simulation is not working correctly. This is probably why Jackal is not responding to the move_base goal commands.

I re-created the issue on my machine. Could you confirm for me, that you can see a ros-warning: [WARN] [1606376918.265182, 0.000000]: Controller Spawner couldn't find the expected controller_manager ROS interface.?

I solved the issue the following way (Couldn't test it on another machine yet, but I'll do that in the afternoon):

  1. Install the X-Server.
  2. Run $ set-variable -name DISPLAY YOUR_IP_ADDRESS:0.0 (of course replace "YOUR_IP_ADDRESS" with your local IP address, which probably starts with 192.168...). You can get your IP address in powershell by executing:
    (
    Get-NetIPConfiguration |
    Where-Object {
        $_.IPv4DefaultGateway -ne $null -and
        $_.NetAdapter.Status -ne "Disconnected"
    }
    ).IPv4Address.IPAddress

    Do not forget the ":0.0" behind the address.

  3. Run $ docker run -it --rm -e DISPLAY=$DISPLAY -p 9090:9090 iros2020.azurecr.io/jackal:latest /bin/bash -c "source /home/catkin_ws/devel/setup.bash && roslaunch ms_jackal_demo jackal_demo.launch"

To be honest I do not fully understand why this approach works on my machine and running it without the X-Server and with "gui:=false" doesn't. If it works for you this way, then you'll get a Gazebo-GUI as a bonus 😊

If for some reason it doesn't work, check if in Gazebo the tires are white and at the center of jackal, and not where they are supposed to be. This happened during the development a few times and had a similar issue. If it still happens on your machine, then we will find another solution.

Best regards, Eric

EricVoll commented 3 years ago

So I just pruned my docker-system to delete all containers and retried the solution above and it worked, while running the command from the "running the demo" section did not.

What I forgot to mention: These are expected errors while launching the container:

PattyUnity commented 3 years ago

Thanks Eric for your prompt reply. I’m wrapping up for the day and will come back to work on this tomorrow.

But for a quick answer to your question about the ros warning; No, I do not see: [WARN] [1606376918.265182, 0.000000]: Controller Spawner couldn't find the expected controller_manager ROS interface.

Below is part of the output on PowerShell:

auto-starting new master process[master]: started with pid [70] ROS_MASTER_URI=http://localhost:11311

setting /run_id to 11ca796e-2f9d-11eb-9919-0242ac110002 process[rosout-1]: started with pid [81] started core service [/rosout] process[rosbridge_websocket-2]: started with pid [88] process[rosapi-3]: started with pid [89] 2020-11-26 04:08:43+0000 [-] Log opened. 2020-11-26 04:08:43+0000 [-] registered capabilities (classes): 2020-11-26 04:08:43+0000 [-] - rosbridge_library.capabilities.call_service.CallService 2020-11-26 04:08:43+0000 [-] - rosbridge_library.capabilities.advertise.Advertise 2020-11-26 04:08:43+0000 [-] - rosbridge_library.capabilities.publish.Publish 2020-11-26 04:08:43+0000 [-] - rosbridge_library.capabilities.subscribe.Subscribe 2020-11-26 04:08:43+0000 [-] - <class 'rosbridge_library.capabilities.defragmentation.Defragment'> 2020-11-26 04:08:43+0000 [-] - rosbridge_library.capabilities.advertise_service.AdvertiseService 2020-11-26 04:08:43+0000 [-] - rosbridge_library.capabilities.service_response.ServiceResponse 2020-11-26 04:08:43+0000 [-] - rosbridge_library.capabilities.unadvertise_service.UnadvertiseService process[file_server-4]: started with pid [94] [ INFO] [1606363723.802419000]: ROSbridge File Server initialized. 2020-11-26 04:08:43+0000 [-] WebSocketServerFactory starting on 9090 2020-11-26 04:08:43+0000 [-] Starting factory <autobahn.twisted.websocket.WebSocketServerFactory object at 0x7fec4c673950> 2020-11-26 04:08:43+0000 [-] [INFO] [1606363723.865214, 0.000000]: Rosbridge WebSocket server started at ws://0.0.0.0:9090 [INFO] [1606363723.912538, 0.000000]: Rosapi started process[move_base-5]: started with pid [100] process[gazebo-6]: started with pid [113] process[gazebo_gui-7]: started with pid [126] [ INFO] [1606363724.351121500]: Finished loading Gazebo ROS API Plugin. [ INFO] [1606363724.352815800]: waitForService: Service [/gazebo/set_physics_properties] has not been advertised, waiting... process[robot_state_publisher-8]: started with pid [167] xcb_connection_has_error() returned true ALSA lib confmisc.c:767:(parse_card) cannot find card '0' ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_card_driver returned error: No such file or directory ALSA lib confmisc.c:392:(snd_func_concat) error evaluating strings ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory ALSA lib confmisc.c:1246:(snd_func_refer) error evaluating name ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM default AL lib: (EE) ALCplaybackAlsa_open: Could not open playback device 'default': No such file or directory [ INFO] [1606363724.537787700]: waitForService: Service [/gazebo/set_physics_properties] is now available. [ INFO] [1606363724.568135700]: Finished loading Gazebo ROS API Plugin. [ INFO] [1606363724.571028500]: waitForService: Service [/gazebo_gui/set_physics_properties] has not been advertised, waiting... [ INFO] [1606363724.573841300, 0.029000000]: Physics dynamic reconfigure ready. process[controller_spawner-9]: started with pid [242] process[ekf_localization-10]: started with pid [337] process[twist_mux-11]: started with pid [344] Segmentation fault [gazebo_gui-7] process has died [pid 126, exit code 139, cmd /opt/ros/melodic/lib/gazebo_ros/gzclient __name:=gazebo_gui __log:=/root/.ros/log/11ca796e-2f9d-11eb-9919-0242ac110002/gazebo_gui-7.log]. log file: /root/.ros/log/11ca796e-2f9d-11eb-9919-0242ac110002/gazebo_gui-7.log [ WARN] [1606363725.083138200, 0.532000000]: Failed to meet update rate! Took 0.53200000000000002842 [ WARN] [1606363725.083942800, 0.533000000]: Failed to meet update rate! Took 0.51300000000000001155 process[urdf_spawner-12]: started with pid [358] [ INFO] [1606363725.756227600, 1.149000000]: Loading gazebo_ros_control plugin [ INFO] [1606363725.756360200, 1.149000000]: Starting gazebo_ros_control plugin in namespace: / [ INFO] [1606363725.757185000, 1.149000000]: gazebo_ros_control plugin is waiting for model URDF in parameter [robot_description] on the ROS param server. [ERROR] [1606363725.866886000, 1.149000000]: No p gain specified for pid. Namespace: /gazebo_ros_control/pid_gains/front_left_wheel [ERROR] [1606363725.867601900, 1.149000000]: No p gain specified for pid. Namespace: /gazebo_ros_control/pid_gains/front_right_wheel [ERROR] [1606363725.868672200, 1.149000000]: No p gain specified for pid. Namespace: /gazebo_ros_control/pid_gains/rear_left_wheel [ERROR] [1606363725.869290200, 1.149000000]: No p gain specified for pid. Namespace: /gazebo_ros_control/pid_gains/rear_right_wheel [ INFO] [1606363725.871953500, 1.149000000]: Loaded gazebo_ros_control. [urdf_spawner-12] process has finished cleanly log file: /root/.ros/log/11ca796e-2f9d-11eb-9919-0242ac110002/urdf_spawner-12.log [ INFO] [1606363726.091001100, 1.317000000]: Controller state will be published at 50Hz.

EricVoll commented 3 years ago

thanks for the output. Two things:

  1. I see that the gazebo_gui process is dying. That usually only happens in docker containers if the X-Server on windows is not started properly, but that shouldn't make a difference. The gui process can die, as long as the real gazebo process doesn't.
  2. I can't see an output from the rosbridge that your hololens / client device ever connected to it. Are you sure that you entered the correct IP-address?
PattyUnity commented 3 years ago

I get the IP address by executing: ( Get-NetIPConfiguration | Where-Object { $.IPv4DefaultGateway -ne $null -and $.NetAdapter.Status -ne "Disconnected" } ).IPv4Address.IPAddress

It starts with 10.137....

Yes, it seems the hololens doesn’t connect to ROS. In MR app log, it keeps saying “Attempting to connect to ws.......”

I am using hololens 1 on Windows 10 with WSL.

EricVoll commented 3 years ago

Ah okey then I think I know how to fix the issue! WSL is a bit weird with local networks in that regard. Please note, that the following is just the result of a few hours of googeling and I am not the best networker out there. You will have to do some port-mapping for that to work if you are not using Docker such that windows forwards the network traffic on this specific port to WSL. In my case I did not have to deactivate the firewall.

This answer explains what you have to execute in an elevated powershell on windows. Select the Port 9090.

EricVoll commented 3 years ago

I'd recommend to use docker though, as it is much less of a pain 👍

PattyUnity commented 3 years ago

Thank you Eric. We will try using Ubuntu and see if we still get the network issue.

roalchaq commented 3 years ago

I had the same issue (including the warning message [WARN] [1607521740.925819, 0.000000]: Controller Spawner couldn't find the expected controller_manager ROS interface.) while running the container on Ubuntu 20.04. To fix it:

$ xhost +si:localuser:root
$ docker run -it --rm -e DISPLAY -p 9090:9090 iros2020.azurecr.io/jackal:latest /bin/bash -c "source /home/catkin_ws/devel/setup.bash && roslaunch ms_jackal_demo jackal_demo.launch"

The gazebo gui process still dies but at least the simulation works and the app runs as expected.