Evaluate ROS - Unity Communication Performance

[gris-martin]

I have a question!

I'm implementing an application where I want Unity to listen for ROS messages on a topic and, as soon as one is received, publish another message on a separate topic. It's important that the response from Unity is as fast as possible, preferably below 5 ms.

I have devised the following setup:

• A Publisher that launches a new thread containing a function with a while(true) loop. The loop publishes a message and then waits for an AutoResetEvent block to be released.
• A Receiver that, when a message is received, releases the Publisher block and allows the loop to go one more round.
• A regular Subscriber using the Receiver as its MessageReceiver.

I'm using ROS on a separate Ubuntu machine to send messages, and then using Wireshark to measure timings. Some observations I've made:

1. The time it takes from publishing the message from ROS, to when a message is received again by ROS, varies, with the shortest time being ~1 ms.
2. The time from 1 varies with the Fixed Timestep (Edit > Project Settings > Time). For large timesteps (> 20 ms, 0.02 s), the largest time seems to vary between 1 ms and the timestep.
3. Decreasing the Fixed Timestep below 20 ms has no further effect on the time described in 1.
4. I have made the same setup in Simulink, using the ROS capabilities from the Robotics System Toolbox. There I've managed to get the response time down to a steady 4-5 ms, so I don't think my issue has to do with ROS. Although Matlab does not use rosbridge as far as I know, so it could be that.

But my question is: What is it that decides when a Subscriber handles a message, and when a Publisher dispatches it? Does it somehow depend on the Fixed Timestep?

Thanks for a great product!

[MartinBischoff]

This is a very interesting evaluation.

To maximize performance, we should stay out of any Unity Event function, except Awake or Start for an initial trigger of a thread-based system that runs in parallel.

We designed the Unity Message Handling Framework to be able to do that.

The MessageReceiver is directly triggered by a Subscriber and can further process the message without waiting for any Unity event function to be called. Correspondingly a MessageProvider triggers a Publisher to immediately forward the message to ROS.

In all our MessageReceiver and MessageProvider implementations (e.g. ImageReceiver) we somehow interact with the Unity scene and thus have to wait for a corresponding event function to get the data into / out of the scene. If no communication with the scene is required, the corresponding MessageReceiver and MessageProvider implementations can go without Update or FixedUpdate calls. See the Message Handling Code Map for an overview on the script interaction.

I have not found any clear information about when or how often calculation time is given to parallel running threads in Unity. It depends on the event & multi-treading architecture and multi-core capabilities of Unity and .NET.

Other than Unity, the only things that slow down the communication on the non-ROS machine are:

• the C# event system which we apply in the message handling framework
• and the JSON (de)serialization which is required to access the message data.

I suggest the following evalutations:

1. To identify the overhead of rosbridge_suite and JSON (de-)serialization only, we can run the same test in a simple C# script with some small adjustment of the code example.

2. We can skip all C# event-related things by modifying the publisher and subscriber scripts to directly send messages via RosSocket.

Please do keep us updated with the tests. If you can share any performance test scripts or Unity projects with us, please do!

[gris-martin]

Thanks for the feedback, I'll get back when I have more data!

[gris-martin]

I have now done some more evaluation with your suggestions in mind. It seems the delays were due to the face that my Provider called its UpdateMessage() method during the FixedUpdate callback. I did some hack to make the Publisher call the UpdateMessage() method instead, and after that I got very fast response times, about 0.6 ms.

I got the same result when I did a pure C# implementation, and when I made new Publishers and Subscribers that did not use the Event system at all.

The files I used to do the evaluation are in my fork if anyone is interested: https://github.com/hassanbot/ros-sharp/tree/performance-evaluation It contains a new project under RosBridgeClient, and two Asset folders under RosSharp. However, the Scene under WithEvents will not work since I did not commit my Publisher hack (which calls an UpdateMessage method of its Provider during StartPublication.

I feel satisfied now, and I think I can make a better solution without hacking already present files. You can close this issue if you don't have anything to add. Thanks for the input before, it really helped me pinpoint the problem :)

[MartinBischoff]

These are very interesting news! 0.6ms is almost ten times faster than your initially required 5ms :+1: Did I understand right that Matworks Robotic System Toolbox performs at 4-5ms on your identical system?

It is good to know that neither the event system nor Unity have a huge impact on the performance when setting up the publishers and subscribers correctly. Thank you for sharing these evaluation results!