search

micro-ROS / micro-ROS-Agent

ROS 2 package using Micro XRCE-DDS Agent.
Apache License 2.0
97 stars 51 forks source link

How to solve the sticky bag problem? #158

Closed vd112 closed 1 year ago

vd112 commented 1 year ago

If I publish 10 packets, I can subscribe to the 10 packets directly.However, after forwarding by Micro ROS Agent, the number of packets received by MCU is less than 10. So I did the following debugging. I sent 10 packets of data and data content is "aaaaaaaaaa+number". Through Wireshark, I found that there were only two packets(RTPS packets) after DDS for the 10 packets I sent.Here are two pictures.The contents of the serializedData packet are correct.

企业微信截图_16575450486470 企业微信截图_16575451552290

By checking agent log, I find that "[==>> DDS <<==]" is two packets of data.So I guess micro Ros Agent didn't disassemble the second RTPS packet.

企业微信截图_16575459078561

I would appreciate it if you could solve my question!

pablogs9 commented 1 year ago

Please provide code for replicating this issue.

vd112 commented 1 year ago

Thank you for your reply. test.zip

pablogs9 commented 1 year ago

I cannot open this file (seems to be corrupted), could you paste here your code, details of your platform, your colcon.meta files and instructions for replicating?

vd112 commented 1 year ago

My platform is Ubuntu 20.04 installed ros2 Foxy version. The following figure shows the code directory structure.

企业微信截图_16575890696270

spubTest_hpp.txt main_cpp.txt spubTest_cpp.txt CMakeLists.txt package_xml.txt The suffix of the above files has been changed to "TXT".You should restore the file name. "spubTest.hpp", "main.cpp", "spubTest.cpp", "CMakeLists.txt", "package.xml".

Acuadros95 commented 1 year ago

Hi @vd112,

I don't think this problem is related to the Agent, I am missing messages using both micro-ROS and Fast DDS subscriptions:

How are you running this on micro-ROS side and what is your use case? Is this solved using reliable entities?

Also, micro-ROS Agent handles incoming DDS data sub messages one by one, it does not matter if they come in a single RTPS message:

[1657709788.646193] debug    | DataReader.cpp     | read_fn                  | [==>> DDS <<==]        | client_key: 0x00000000, len: 16, data: 
0000: 0C 00 00 00 61 61 61 61 61 61 61 61 61 61 33 00
[1657709788.646274] debug    | DataReader.cpp     | read_fn                  | [==>> DDS <<==]        | client_key: 0x00000000, len: 16, data: 
0000: 0C 00 00 00 61 61 61 61 61 61 61 61 61 61 35 00
[1657709788.646290] debug    | DataReader.cpp     | read_fn                  | [==>> DDS <<==]        | client_key: 0x00000000, len: 16, data: 
0000: 0C 00 00 00 61 61 61 61 61 61 61 61 61 61 36 00
[1657709788.646290] debug    | UDPv4AgentLinux.cpp | send_message             | [** <<UDP>> **]        | client_key: 0x66873B66, len: 28, data: 
0000: 81 01 00 00 09 01 14 00 00 0E 00 06 0C 00 00 00 61 61 61 61 61 61 61 61 61 61 33 00
[1657709788.646307] debug    | DataReader.cpp     | read_fn                  | [==>> DDS <<==]        | client_key: 0x00000000, len: 16, data: 
0000: 0C 00 00 00 61 61 61 61 61 61 61 61 61 61 37 00
[1657709788.646331] debug    | UDPv4AgentLinux.cpp | send_message             | [** <<UDP>> **]        | client_key: 0x66873B66, len: 28, data: 
0000: 81 01 01 00 09 01 14 00 00 0E 00 06 0C 00 00 00 61 61 61 61 61 61 61 61 61 61 35 00
[1657709788.646336] debug    | DataReader.cpp     | read_fn                  | [==>> DDS <<==]        | client_key: 0x00000000, len: 16, data: 
0000: 0C 00 00 00 61 61 61 61 61 61 61 61 61 61 38 00
[1657709788.646343] debug    | UDPv4AgentLinux.cpp | send_message             | [** <<UDP>> **]        | client_key: 0x66873B66, len: 28, data: 
0000: 81 01 02 00 09 01 14 00 00 0E 00 06 0C 00 00 00 61 61 61 61 61 61 61 61 61 61 36 00
[1657709788.646348] debug    | DataReader.cpp     | read_fn                  | [==>> DDS <<==]        | client_key: 0x00000000, len: 16, data: 
0000: 0C 00 00 00 61 61 61 61 61 61 61 61 61 61 39 00
[1657709788.646351] debug    | UDPv4AgentLinux.cpp | send_message             | [** <<UDP>> **]        | client_key: 0x66873B66, len: 28, data: 
0000: 81 01 03 00 09 01 14 00 00 0E 00 06 0C 00 00 00 61 61 61 61 61 61 61 61 61 61 37 00
[1657709788.646359] debug    | UDPv4AgentLinux.cpp | send_message             | [** <<UDP>> **]        | client_key: 0x66873B66, len: 28, data: 
0000: 81 01 04 00 09 01 14 00 00 0E 00 06 0C 00 00 00 61 61 61 61 61 61 61 61 61 61 38 00
[1657709788.646366] debug    | UDPv4AgentLinux.cpp | send_message             | [** <<UDP>> **]        | client_key: 0x66873B66, len: 28, data: 
0000: 81 01 05 00 09 01 14 00 00 0E 00 06 0C 00 00 00 61 61 61 61 61 61 61 61 61 61 39 00
[1657709788.646390] debug    | DataReader.cpp     | read_fn                  | [==>> DDS <<==]        | client_key: 0x00000000, len: 20, data: 
0000: 0D 00 00 00 61 61 61 61 61 61 61 61 61 61 31 30 00 00 00 00
[1657709788.646434] debug    | UDPv4AgentLinux.cpp | send_message             | [** <<UDP>> **]        | client_key: 0x66873B66, len: 32, data: 
0000: 81 01 06 00 09 01 18 00 00 0E 00 06 0D 00 00 00 61 61 61 61 61 61 61 61 61 61 31 30 00 00 00 00
vd112 commented 1 year ago

Hi @Acuadros95 , Thanks for your response. The result of your experiment is the same as that of mine.But I think "ros2 topic echo /test_spub" is not a good way to verify.So I created a Subscription project code similar to Publication.Running the Subscription project will see that it can receive all data sent by the Publication.To solve these problems, I used Wireshark to capture and analyze packets.The captured data discovery Publication sent all the data.So the Subscription project code I created accepts all data.So I think "ROS2 topic echo/test_spub" is not a good way to verify. Here is my Subscription project file.Please let me know if you can't open this file. sub_test.zip I'm sorry I don't agree with that”Also, micro-ROS Agent handles incoming DDS data sub messages one by one, it does not matter if they come in a single RTPS message“.I did a lot of experiments and it turns out that if an RTPS packet has multiple data it can only parse one data.I can guess that the DDS in your reply to my example sent only 7 RTPS packets, but the 7 packets contained all the data.“aaaaaaaaaa1”,“aaaaaaaaaa2” and "aaaaaaaaaa3" in the first RTPS packet."aaaaaaaaaa4" and "aaaaaaaaaa5" in the second RTPS packet.You can use wireshark to verify my guess. So my question now is if "aaaaaaaaaa1", "aaaaaaaaaa2" and "aaaaaaaaaa3" are in the same RTPS packet, why can micro-ros Agent only receive "aaaaaaaaaa3". I'm going to do an experiment today using Reliable.I anticipate your reply sincerely.

Acuadros95 commented 1 year ago

Reliability should solve this, but you are right, this behavior does not make much sense. I am going to take a good look into this.

vd112 commented 1 year ago

Thank you very much.I look forward to your research results.

Acuadros95 commented 1 year ago

@vd112, got it!

The problem is not on the Agent, but in the micro-ROS subscriber QoS configuration.

For best effort subscribers we set the history depth to 5 by default (rmw_qos_profile_sensor_data), and as you are sending 10 messages in a burst, the history will drop some of them as it only has space for 5 data messages.

You can modify this configuration on the subscription init method:

rmw_qos_profile_t subscriber_qos = rmw_qos_profile_sensor_data;
subscriber_qos.depth = 10;

// create subscriber
RCCHECK(rclc_subscription_init(
    &subscriber,
    &node,
    ROSIDL_GET_MSG_TYPE_SUPPORT(std_msgs, msg, String),
    "/test_spub",
    &subscriber_qos));

Please test it out!

Acuadros95 commented 1 year ago

Could you share your micro-ROS code along with your setup? (Board, build system used, ROS Distro, ...)

Also, please share your Agent debug log (-v6) for this test.

vd112 commented 1 year ago

Hi, @Acuadros95, I'm sorry that I modified the "Subscriber_qos.depth = 10" and the result is the same as before. I tried to solve this problem in a reliable way but the result was the same as before.Therefore, I think the main problem is how to deal with the agent when it receives a RTPS package containing multiple data. The MCU I use is STM32F429 connected to RK3399 through serial port.Rk3399 soc runs ubuntu 20.04.The ros version is ROS2 Foxy.Agent configuration is the following files "microros_lib.cmake" and "microros_lib.meta". agent_log.txt microros_lib_cmake.txt microros_lib_meta.txt

Acuadros95 commented 1 year ago

There are a lot of things going on on that log. Please try to replicate the error with a minimum replicable micro-ROS example.

Also remember to include this minimum micro-ROS code, so we can replicate, debug and find a fix for the problem on our side.

vd112 commented 1 year ago

Hi, @Acuadros95, I want to know if changing QOS subscriber configuration Agent can forward all publisher data in this experiment？I'm shrinking the log. https://github.com/micro-ROS/micro-ROS-Agent/issues/158#issuecomment-1183079272

Acuadros95 commented 1 year ago

QOS subscriber configuration Agent

What do you mean exactly? The Agent subscribers uses the same QoS as the micro-ROS subscriber, that is why you should get the data after the depth or reliability modifications.

There may be other factors here, but the Agent should get all the data.

vd112 commented 1 year ago

Sorry, my engineering environment is complicated.It is not easy to extract the minimum micro-ros code. I think you can solve this problem on the platform you've already established(This platform: https://github.com/micro-ROS/micro-ROS-Agent/issues/158#issuecomment-1183079272 ).Maybe my QOS configuration is ok.The main problem may be Agent parsing RTPS packets. Thank you!

Acuadros95 commented 1 year ago

Could you share your exact micro-ROS subscriber initialization code?

We already checked the Agent with a simple test and it looks OK after modifying the depth as explained here: https://github.com/micro-ROS/micro-ROS-Agent/issues/158#issuecomment-1184165298

vd112 commented 1 year ago

I'm cleaning up the initialization process. Wait reply.

vd112 commented 1 year ago

I initialized subscriber using the function “rclc_subscription_init_best_effort”.The details are in the following file.Please start at the bottom. subscription_init_c.txt

Acuadros95 commented 1 year ago

I dont understand your attached file, are you re implementing the rclc methods? Why is the implementation of rcl_subscription_init there?

vd112 commented 1 year ago

Yes,it is.The MCU I use is STM32F429 connected to RK3399 through serial port.MCU code is written in C language.MCU and Linux platform RK3399 use Micro ROS Agent to publish and subscribe messages.