search

micro-ROS / zephyr_apps

Sample applications for Zephyr + micro-ROS
Apache License 2.0
19 stars 13 forks source link

Missing Service example #4

Closed gramss closed 2 years ago

gramss commented 4 years ago

Created a new issue to not mix it up with #3

I tried to implement the example from here: https://github.com/micro-ROS/micro-ROS-demos/tree/dashing/rcl/addtwoints_server

This is my approach: https://github.com/gramss/zephyr_apps/tree/dashing/apps/serial_service_addTwoInts_example

I took the main.c 1:1 and added some printf to see if the main.c is run. After enabling printk=y and the banner I only got the banner and no output from my main nor an error from the serial device like here: https://github.com/micro-ROS/zephyr_apps/pull/2#discussion_r426737007

But I'm quite sure that I need to add the CMakeList parameters as well.. But I'm not aware how I can add these rcl related CMakeList parameters into the zephyr CMakeList... For example making rcl aware of the srv etc..

each zephyr build gives me errors like this:

--- stderr: rosidl_generator_dds_idl                                                                                             
CMake Warning:
  Manually-specified variables were not used by the project:

    BUILD_SHARED_LIBS
    THIRDPARTY

THIRDPARTY is nearly always there and sometime there is some other flag.. How can I seperate those erros from missing dependency checks I know from ros2 programming on my linux machine.. Package xyz not found and so on..

As falsely described in #3 I get no output from the main.c even after enableing printk=y. It feels like the main is never compiled successfully and an empty image

pablogs9 commented 4 years ago

Why are you enabling printk? I guess that this printk will interfere with the micro-ROS serial communication as we talked here: https://github.com/micro-ROS/micro-ros-build/issues/115

Have you thought in using gdb for debugging your app step by step? For us is really useful this kind of debug to check where are the app failing.

pablogs9 commented 4 years ago

You can check another example os services here: https://github.com/micro-ROS/freertos_apps/blob/e0e857942449b786829c20e423a6fa0ab4600221/apps/crazyflie_uros_sample/app.c#L336

gramss commented 4 years ago

I enabled printk because when I do I still see output made by the serial stack. the agent might be failing at this point. But before I let the agent work I revert the enabled printk.

This helped me for example to get this error: https://github.com/micro-ROS/zephyr_apps/pull/2#discussion_r426737007

Yes. I should start working with gdb. With west debugserveryou can easily setup an gdb server. The problem with this is that the whole build system is not inside an IDE. I failed setting it up with PlatformIO. Do you have a recommended setup to use gbd with your build setup? I would prefer not to learn interacting with gdb on the command line. It is a powerful tool when I can inject debug symbols and have my source code loaded.

I would love to use it! And I'm familiar operating a debugger, but the "closed" build setup makes it difficult for me to set it up on my own.

gramss commented 4 years ago

You can check another example os services here: https://github.com/micro-ROS/freertos_apps/blob/e0e857942449b786829c20e423a6fa0ab4600221/apps/crazyflie_uros_sample/app.c#L336

Thank you. This helped to make sure that I only need to setup 1 node and 1 server in my app-colcon.meta

gramss commented 4 years ago

@pablogs9 I'm not seeing any obvious flaws in my code. It is updated to work with the USART routine.. Maybe a service alone was never tested. Is another publisher required or does the service itself does not have the capability to register itself..

I would appreciate if you could help me here. Or if somebody can point me out how to use a gdb. I'm currently failing to follow the gdb setup from Nuttx with zephyr. I'm unable to find the right arm-none-eabi-gdbfor openocd ... there is no such file existing in my whole system. I even cloned and installed the zypher-openocd git. Only to find out that it is already included in the zypherprojects package and used by west to flash my board since the beginning..

pablogs9 commented 4 years ago

Please check this two PR and their branches, I have my add two ints working here:

https://github.com/micro-ROS/rmw-microxrcedds/pull/63 https://github.com/micro-ROS/zephyr_apps/pull/5

It should work for you with the same configuration as the publishers (serial)

gramss commented 4 years ago

I checked out your PRs.

It is now listed as a service in my ros2 network. When I run the demo_node client on my linux host I get the following output:

$ros2 run demo_nodes_cpp add_two_ints_client
[INFO] [add_two_ints_client]: service not available, waiting again...
[INFO] [add_two_ints_client]: service not available, waiting again...
[INFO] [add_two_ints_client]: service not available, waiting again...
[INFO] [add_two_ints_client]: service not available, waiting again...
[INFO] [add_two_ints_client]: service not available, waiting again...
[INFO] [add_two_ints_client]: service not available, waiting again...
^C[INFO] [rclcpp]: signal_handler(signal_value=2)
[ERROR] [add_two_ints_client]: Interrupted while waiting for the service. Exiting.
$ ros2 run demo_nodes_cpp add_two_ints_client
[INFO] [add_two_ints_client]: service not available, waiting again...
[INFO] [add_two_ints_client]: service not available, waiting again...
[INFO] [add_two_ints_client]: service not available, waiting again...
[INFO] [add_two_ints_client]: service not available, waiting again...
[INFO] [add_two_ints_client]: service not available, waiting again...
[INFO] [add_two_ints_client]: service not available, waiting again...
^C[INFO] [rclcpp]: signal_handler(signal_value=2)
[ERROR] [add_two_ints_client]: Interrupted while waiting for the service. Exiting.

This is displayed by the mirco-agent:

$ ros2 run micro_ros_agent micro_ros_agent serial --dev /dev/ttyACM0 -v6
Press CTRL+C to exit
[1589905646.987796] info     | TermiosAgentLinux.cpp | init                     | running...             | fd: 3
[1589905646.988116] info     | Root.cpp           | set_verbose_level        | logger setup           | verbose_level: 6
[1589905647.987203] info     | Root.cpp           | create_client            | create                 | client_key: 0xDEADBEEF, session_id: 0x81
[1589905647.987294] info     | SessionManager.hpp | establish_session        | session established    | client_key: 0x3735928559, address: 1
[1589905647.987475] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 19, data: 
0000: 81 00 00 00 04 01 0B 00 00 00 58 52 43 45 01 00 01 0F 00
[1589905648.002599] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 108, data: 
0000: 81 80 00 00 01 05 64 00 00 0A 00 01 01 02 00 00 56 00 00 00 3C 64 64 73 3E 3C 70 61 72 74 69 63
0020: 69 70 61 6E 74 3E 3C 72 74 70 73 3E 3C 6E 61 6D 65 3E 61 64 64 74 6F 77 69 6E 74 73 5F 73 65 72
0040: 76 65 72 5F 72 63 6C 3C 2F 6E 61 6D 65 3E 3C 2F 72 74 70 73 3E 3C 2F 70 61 72 74 69 63 69 70 61
0060: 6E 74 3E 3C 2F 64 64 73 3E 00 00 00
[1589905648.005922] debug    | ProxyClient.cpp    | create_participant       | participant created    | client_key: 0xDEADBEEF, participant_id: 0x000(1)
[1589905648.006097] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 14, data: 
0000: 81 80 00 00 05 01 06 00 00 0A 00 01 00 00
[1589905648.006132] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 01 00 00 00 80
[1589905648.006357] trace    | FastEntities.cpp   | onParticipantDiscovery   | matched                | entity_id: 0.0.1.c1, guid_prefix: 1.f.ed.be.3b.75.0.0.1.0.0.0
[1589905648.006509] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 00 00 00 00 80
[1589905648.006716] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 01 00 00 00 80
[1589905648.007621] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 00 00 00 00 80
[1589905648.007812] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 01 00 00 00 80
[1589905648.195117] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 00 00 00 00 80
[1589905648.304145] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 00 00 00 01 80
[1589905648.304723] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 14, data: 
0000: 81 80 00 00 05 01 06 00 00 0A 00 01 00 00
[1589905648.311286] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 01 00 00 00 80
[1589905648.322731] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 124, data: 
0000: 81 80 01 00 01 05 73 00 00 0B 00 12 02 02 00 00 65 00 00 00 3C 64 64 73 3E 3C 74 6F 70 69 63 3E
0020: 3C 6E 61 6D 65 3E 72 74 2F 72 6F 73 6F 75 74 3C 2F 6E 61 6D 65 3E 3C 64 61 74 61 54 79 70 65 3E
0040: 72 63 6C 5F 69 6E 74 65 72 66 61 63 65 73 3A 3A 6D 73 67 3A 3A 64 64 73 5F 3A 3A 4C 6F 67 5F 3C
0060: 2F 64 61 74 61 54 79 70 65 3E 3C 2F 74 6F 70 69 63 3E 3C 2F 64 64 73 3E 00 00 01 00
[1589905648.322996] debug    | ProxyClient.cpp    | create_topic             | topic created          | client_key: 0xDEADBEEF, topic_id: 0x001(2), participant_id: 0x000(1)
[1589905648.323172] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 14, data: 
0000: 81 80 01 00 05 01 06 00 00 0B 00 12 00 00
[1589905648.323216] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 02 00 00 00 80
[1589905648.326799] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 01 00 01 00 80
[1589905648.327030] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 02 00 00 00 80
[1589905648.328672] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 02 00 00 00 80
[1589905648.362517] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 388, data: 
0000: 81 80 02 00 01 05 25 00 00 0C 00 23 03 02 00 00 17 00 00 00 3C 70 75 62 6C 69 73 68 65 72 20 6E
0020: 61 6D 65 3D 22 32 5F 33 22 3E 00 00 01 00 00 00 01 05 4E 01 00 0D 00 35 05 02 00 00 40 01 00 00
0040: 3C 64 64 73 3E 3C 64 61 74 61 5F 77 72 69 74 65 72 3E 3C 68 69 73 74 6F 72 79 4D 65 6D 6F 72 79
0060: 50 6F 6C 69 63 79 3E 50 52 45 41 4C 4C 4F 43 41 54 45 44 5F 57 49 54 48 5F 52 45 41 4C 4C 4F 43
0080: 3C 2F 68 69 73 74 6F 72 79 4D 65 6D 6F 72 79 50 6F 6C 69 63 79 3E 3C 71 6F 73 3E 3C 72 65 6C 69
00A0: 61 62 69 6C 69 74 79 3E 3C 6B 69 6E 64 3E 52 45 4C 49 41 42 4C 45 3C 2F 6B 69 6E 64 3E 3C 2F 72
00C0: 65 6C 69 61 62 69 6C 69 74 79 3E 3C 2F 71 6F 73 3E 3C 74 6F 70 69 63 3E 3C 6B 69 6E 64 3E 4E 4F
00E0: 5F 4B 45 59 3C 2F 6B 69 6E 64 3E 3C 6E 61 6D 65 3E 72 74 2F 72 6F 73 6F 75 74 3C 2F 6E 61 6D 65
0100: 3E 3C 64 61 74 61 54 79 70 65 3E 72 63 6C 5F 69 6E 74 65 72 66 61 63 65 73 3A 3A 6D 73 67 3A 3A
0120: 64 64 73 5F 3A 3A 4C 6F 67 5F 3C 2F 64 61 74 61 54 79 70 65 3E 3C 68 69 73 74 6F 72 79 51 6F 73
0140: 3E 3C 6B 69 6E 64 3E 4B 45 45 50 5F 41 4C 4C 3C 2F 6B 69 6E 64 3E 3C 2F 68 69 73 74 6F 72 79 51
0160: 6F 73 3E 3C 2F 74 6F 70 69 63 3E 3C 2F 64 61 74 61 5F 77 72 69 74 65 72 3E 3C 2F 64 64 73 3E 00
0180: 00 23 00 00
[1589905648.362732] debug    | ProxyClient.cpp    | create_publisher         | publisher created      | client_key: 0xDEADBEEF, publisher_id: 0x002(3), participant_id: 0x000(1)
[1589905648.362879] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 14, data: 
0000: 81 80 02 00 05 01 06 00 00 0C 00 23 00 00
[1589905648.363198] debug    | ProxyClient.cpp    | create_datawriter        | datawriter created     | client_key: 0xDEADBEEF, datawriter_id: 0x003(5), publisher_id: 0x002(3)
[1589905648.363325] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 14, data: 
0000: 81 80 03 00 05 01 06 00 00 0D 00 35 00 00
[1589905648.363355] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 03 00 00 00 80
[1589905648.366509] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 01 00 02 00 80
[1589905648.366717] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 03 00 00 00 80
[1589905648.396041] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 02 00 03 00 80
[1589905648.401289] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 01 00 02 00 80
[1589905648.401789] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 03 00 00 00 80
[1589905648.401857] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 02 00 00 03 80
[1589905648.402158] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 14, data: 
0000: 81 80 02 00 05 01 06 00 00 0C 00 23 00 00
[1589905648.402202] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 14, data: 
0000: 81 80 03 00 05 01 06 00 00 0D 00 35 00 00
[1589905648.510767] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 03 00 00 01 80
[1589905648.511009] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 14, data: 
0000: 81 80 03 00 05 01 06 00 00 0D 00 35 00 00
[1589905648.518206] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 04 00 00 00 80
[1589905648.547543] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 348, data: 
0000: 81 80 03 00 01 05 52 01 00 0E 00 48 08 02 00 00 44 01 00 00 3C 64 64 73 3E 3C 72 65 70 6C 69 65
0020: 72 20 70 72 6F 66 69 6C 65 5F 6E 61 6D 65 3D 22 34 5F 38 22 20 73 65 72 76 69 63 65 5F 6E 61 6D
0040: 65 3D 22 2F 61 64 64 74 77 6F 69 6E 74 73 22 20 72 65 71 75 65 73 74 5F 74 79 70 65 3D 22 65 78
0060: 61 6D 70 6C 65 5F 69 6E 74 65 72 66 61 63 65 73 3A 3A 73 72 76 3A 3A 64 64 73 5F 3A 3A 41 64 64
0080: 54 77 6F 49 6E 74 73 5F 52 65 71 75 65 73 74 5F 22 20 72 65 70 6C 79 5F 74 79 70 65 3D 22 65 78
00A0: 61 6D 70 6C 65 5F 69 6E 74 65 72 66 61 63 65 73 3A 3A 73 72 76 3A 3A 64 64 73 5F 3A 3A 41 64 64
00C0: 54 77 6F 49 6E 74 73 5F 52 65 73 70 6F 6E 73 65 5F 22 3E 3C 72 65 71 75 65 73 74 5F 74 6F 70 69
00E0: 63 5F 6E 61 6D 65 3E 72 71 2F 61 64 64 74 77 6F 69 6E 74 73 52 65 71 75 65 73 74 3C 2F 72 65 71
0100: 75 65 73 74 5F 74 6F 70 69 63 5F 6E 61 6D 65 3E 3C 72 65 70 6C 79 5F 74 6F 70 69 63 5F 6E 61 6D
0120: 65 3E 72 72 2F 61 64 64 74 77 6F 69 6E 74 73 52 65 70 6C 79 3C 2F 72 65 70 6C 79 5F 74 6F 70 69
0140: 63 5F 6E 61 6D 65 3E 3C 2F 72 65 70 6C 69 65 72 3E 3C 2F 64 64 73 3E 00 00 01 00 00
[1589905648.548075] debug    | ProxyClient.cpp    | create_replier           | replier created        | client_key: 0xDEADBEEF, requester_id: 0x004(7), participant_id: 0x000(1)
[1589905648.548187] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 14, data: 
0000: 81 80 04 00 05 01 06 00 00 0E 00 48 00 00
[1589905648.548220] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 04 00 00 00 80
[1589905648.553540] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 03 00 03 00 80
[1589905648.553726] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 04 00 00 00 80
[1589905648.555534] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 03 00 03 00 80
[1589905648.555689] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 04 00 00 00 80
[1589905648.597053] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 04 00 04 00 80
[1589905648.601726] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 03 00 03 00 80
[1589905648.602032] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 04 00 00 00 80
[1589905648.708200] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 04 00 00 01 80
[1589905648.708752] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 14, data: 
0000: 81 80 04 00 05 01 06 00 00 0E 00 48 00 00
[1589905648.715325] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 05 00 00 00 80
[1589905648.721027] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 24, data: 
0000: 81 80 04 00 08 01 10 00 00 0F 00 48 80 00 00 01 FF FF 00 00 00 00 00 00
[1589905648.721531] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 05 00 00 00 80
[1589905648.724713] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 04 00 04 00 80
[1589905648.724990] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 05 00 00 00 80
[1589905648.828601] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 04 00 04 00 80
[1589905648.828967] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 05 00 00 00 80
[1589905654.278846] trace    | FastEntities.cpp   | onParticipantDiscovery   | matched                | entity_id: 0.0.1.c1, guid_prefix: 1.f.ed.be.94.76.0.0.1.0.0.0
[1589905664.073626] trace    | FastEntities.cpp   | onParticipantDiscovery   | unmatched              | entity_id: 0.0.1.c1, guid_prefix: 1.f.ed.be.94.76.0.0.1.0.0.0
[1589905705.245720] trace    | FastEntities.cpp   | onParticipantDiscovery   | matched                | entity_id: 0.0.1.c1, guid_prefix: 1.f.ed.be.a7.76.0.0.1.0.0.0
[1589905709.887437] trace    | FastEntities.cpp   | onParticipantDiscovery   | unmatched              | entity_id: 0.0.1.c1, guid_prefix: 1.f.ed.be.a7.76.0.0.1.0.0.0
[1589905718.747363] trace    | FastEntities.cpp   | onParticipantDiscovery   | matched                | entity_id: 0.0.1.c1, guid_prefix: 1.f.ed.be.2.77.0.0.1.0.0.0
[1589905729.131590] trace    | FastEntities.cpp   | onParticipantDiscovery   | matched                | entity_id: 0.0.1.c1, guid_prefix: 1.f.ed.be.0.79.0.0.1.0.0.0
[1589905732.021891] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 24, data: 
0000: 80 00 00 00 00 01 10 00 58 52 43 45 01 00 01 0F DE AD BE EF 81 00 FC 01
[1589905732.022074] info     | SessionManager.hpp | establish_session        | session re-established | client_key: 0x3735928559, address: 1
[1589905732.022213] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 19, data: 
0000: 81 00 00 00 04 01 0B 00 00 00 58 52 43 45 01 00 01 0F 00
[1589905732.036576] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 108, data: 
0000: 81 80 00 00 01 05 64 00 00 0A 00 01 01 02 00 00 56 00 00 00 3C 64 64 73 3E 3C 70 61 72 74 69 63
0020: 69 70 61 6E 74 3E 3C 72 74 70 73 3E 3C 6E 61 6D 65 3E 61 64 64 74 6F 77 69 6E 74 73 5F 73 65 72
0040: 76 65 72 5F 72 63 6C 3C 2F 6E 61 6D 65 3E 3C 2F 72 74 70 73 3E 3C 2F 70 61 72 74 69 63 69 70 61
0060: 6E 74 3E 3C 2F 64 64 73 3E 00 00 00
[1589905732.041548] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 00 00 00 00 80
[1589905732.046632] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 00 00 00 00 80
[1589905732.058840] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 00 00 00 00 80
[1589905732.075037] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 00 00 00 00 80
[1589905732.113344] debug    | ProxyClient.cpp    | delete_object_unlock     | object deleted         | client_key: 0xDEADBEEF, object_id: 0x0000
[1589905732.115651] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 00 00 00 00 80
[1589905732.116104] debug    | ProxyClient.cpp    | create_participant       | participant created    | client_key: 0xDEADBEEF, participant_id: 0x000(1)
[1589905732.116248] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 14, data: 
0000: 81 80 00 00 05 01 06 00 00 0A 00 01 00 00
[1589905732.116313] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 01 00 00 00 80
[1589905732.116342] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 01 00 00 00 80
[1589905732.116410] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 01 00 00 00 80
[1589905732.116428] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 01 00 00 00 80
[1589905732.116441] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 01 00 00 00 80
[1589905732.116452] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 01 00 00 00 80
[1589905732.117442] trace    | FastEntities.cpp   | onParticipantDiscovery   | matched                | entity_id: 0.0.1.c1, guid_prefix: 1.f.ed.be.3b.75.0.0.1.0.0.0
[1589905732.117542] trace    | FastEntities.cpp   | onParticipantDiscovery   | matched                | entity_id: 0.0.1.c1, guid_prefix: 1.f.ed.be.0.79.0.0.1.0.0.0
[1589905732.119601] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 00 00 00 00 80
[1589905732.119800] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 01 00 00 00 80
[1589905732.122531] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 01 00 00 00 80
[1589905732.186843] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 124, data: 
0000: 81 80 01 00 01 05 73 00 00 0B 00 12 02 02 00 00 65 00 00 00 3C 64 64 73 3E 3C 74 6F 70 69 63 3E
0020: 3C 6E 61 6D 65 3E 72 74 2F 72 6F 73 6F 75 74 3C 2F 6E 61 6D 65 3E 3C 64 61 74 61 54 79 70 65 3E
0040: 72 63 6C 5F 69 6E 74 65 72 66 61 63 65 73 3A 3A 6D 73 67 3A 3A 64 64 73 5F 3A 3A 4C 6F 67 5F 3C
0060: 2F 64 61 74 61 54 79 70 65 3E 3C 2F 74 6F 70 69 63 3E 3C 2F 64 64 73 3E 00 00 01 00
[1589905732.187219] debug    | ProxyClient.cpp    | create_topic             | topic created          | client_key: 0xDEADBEEF, topic_id: 0x001(2), participant_id: 0x000(1)
[1589905732.187423] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 14, data: 
0000: 81 80 01 00 05 01 06 00 00 0B 00 12 00 00
[1589905732.187479] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 02 00 00 00 80
[1589905732.192152] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 00 00 01 00 80
[1589905732.192473] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 02 00 00 00 80
[1589905732.193661] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 02 00 00 00 80
[1589905732.332448] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 388, data: 
0000: 81 80 02 00 01 05 25 00 00 0C 00 23 03 02 00 00 17 00 00 00 3C 70 75 62 6C 69 73 68 65 72 20 6E
0020: 61 6D 65 3D 22 32 5F 33 22 3E 00 00 01 00 00 00 01 05 4E 01 00 0D 00 35 05 02 00 00 40 01 00 00
0040: 3C 64 64 73 3E 3C 64 61 74 61 5F 77 72 69 74 65 72 3E 3C 68 69 73 74 6F 72 79 4D 65 6D 6F 72 79
0060: 50 6F 6C 69 63 79 3E 50 52 45 41 4C 4C 4F 43 41 54 45 44 5F 57 49 54 48 5F 52 45 41 4C 4C 4F 43
0080: 3C 2F 68 69 73 74 6F 72 79 4D 65 6D 6F 72 79 50 6F 6C 69 63 79 3E 3C 71 6F 73 3E 3C 72 65 6C 69
00A0: 61 62 69 6C 69 74 79 3E 3C 6B 69 6E 64 3E 52 45 4C 49 41 42 4C 45 3C 2F 6B 69 6E 64 3E 3C 2F 72
00C0: 65 6C 69 61 62 69 6C 69 74 79 3E 3C 2F 71 6F 73 3E 3C 74 6F 70 69 63 3E 3C 6B 69 6E 64 3E 4E 4F
00E0: 5F 4B 45 59 3C 2F 6B 69 6E 64 3E 3C 6E 61 6D 65 3E 72 74 2F 72 6F 73 6F 75 74 3C 2F 6E 61 6D 65
0100: 3E 3C 64 61 74 61 54 79 70 65 3E 72 63 6C 5F 69 6E 74 65 72 66 61 63 65 73 3A 3A 6D 73 67 3A 3A
0120: 64 64 73 5F 3A 3A 4C 6F 67 5F 3C 2F 64 61 74 61 54 79 70 65 3E 3C 68 69 73 74 6F 72 79 51 6F 73
0140: 3E 3C 6B 69 6E 64 3E 4B 45 45 50 5F 41 4C 4C 3C 2F 6B 69 6E 64 3E 3C 2F 68 69 73 74 6F 72 79 51
0160: 6F 73 3E 3C 2F 74 6F 70 69 63 3E 3C 2F 64 61 74 61 5F 77 72 69 74 65 72 3E 3C 2F 64 64 73 3E 00
0180: 00 23 00 00
[1589905732.332699] debug    | ProxyClient.cpp    | create_publisher         | publisher created      | client_key: 0xDEADBEEF, publisher_id: 0x002(3), participant_id: 0x000(1)
[1589905732.332789] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 14, data: 
0000: 81 80 02 00 05 01 06 00 00 0C 00 23 00 00
[1589905732.333217] debug    | ProxyClient.cpp    | create_datawriter        | datawriter created     | client_key: 0xDEADBEEF, datawriter_id: 0x003(5), publisher_id: 0x002(3)
[1589905732.333394] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 14, data: 
0000: 81 80 03 00 05 01 06 00 00 0D 00 35 00 00
[1589905732.333426] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 03 00 00 00 80
[1589905732.339234] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 03 00 00 00 80
[1589905732.431671] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 03 00 03 00 80
[1589905732.436959] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 00 00 02 00 80
[1589905732.437247] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 03 00 00 00 80
[1589905732.540277] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 03 00 00 01 80
[1589905732.540758] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 14, data: 
0000: 81 80 03 00 05 01 06 00 00 0D 00 35 00 00
[1589905732.548154] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 04 00 00 00 80
[1589905732.579398] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 348, data: 
0000: 81 80 03 00 01 05 52 01 00 0E 00 48 08 02 00 00 44 01 00 00 3C 64 64 73 3E 3C 72 65 70 6C 69 65
0020: 72 20 70 72 6F 66 69 6C 65 5F 6E 61 6D 65 3D 22 34 5F 38 22 20 73 65 72 76 69 63 65 5F 6E 61 6D
0040: 65 3D 22 2F 61 64 64 74 77 6F 69 6E 74 73 22 20 72 65 71 75 65 73 74 5F 74 79 70 65 3D 22 65 78
0060: 61 6D 70 6C 65 5F 69 6E 74 65 72 66 61 63 65 73 3A 3A 73 72 76 3A 3A 64 64 73 5F 3A 3A 41 64 64
0080: 54 77 6F 49 6E 74 73 5F 52 65 71 75 65 73 74 5F 22 20 72 65 70 6C 79 5F 74 79 70 65 3D 22 65 78
00A0: 61 6D 70 6C 65 5F 69 6E 74 65 72 66 61 63 65 73 3A 3A 73 72 76 3A 3A 64 64 73 5F 3A 3A 41 64 64
00C0: 54 77 6F 49 6E 74 73 5F 52 65 73 70 6F 6E 73 65 5F 22 3E 3C 72 65 71 75 65 73 74 5F 74 6F 70 69
00E0: 63 5F 6E 61 6D 65 3E 72 71 2F 61 64 64 74 77 6F 69 6E 74 73 52 65 71 75 65 73 74 3C 2F 72 65 71
0100: 75 65 73 74 5F 74 6F 70 69 63 5F 6E 61 6D 65 3E 3C 72 65 70 6C 79 5F 74 6F 70 69 63 5F 6E 61 6D
0120: 65 3E 72 72 2F 61 64 64 74 77 6F 69 6E 74 73 52 65 70 6C 79 3C 2F 72 65 70 6C 79 5F 74 6F 70 69
0140: 63 5F 6E 61 6D 65 3E 3C 2F 72 65 70 6C 69 65 72 3E 3C 2F 64 64 73 3E 00 00 01 00 00
[1589905732.579865] debug    | ProxyClient.cpp    | create_replier           | replier created        | client_key: 0xDEADBEEF, requester_id: 0x004(7), participant_id: 0x000(1)
[1589905732.579964] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 14, data: 
0000: 81 80 04 00 05 01 06 00 00 0E 00 48 00 00
[1589905732.579986] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 04 00 00 00 80
[1589905732.583367] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 03 00 03 00 80
[1589905732.583547] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 04 00 00 00 80
[1589905732.585456] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 05 00 00 00 80
[1589905732.587425] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 24, data: 
0000: 81 80 04 00 08 01 10 00 00 0F 00 48 80 00 00 01 FF FF 00 00 00 00 00 00
[1589905732.587700] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 05 00 00 00 80
[1589905732.690849] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 03 00 04 00 80
[1589905732.691217] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 05 00 00 00 80
[1589905735.146954] trace    | FastEntities.cpp   | onParticipantDiscovery   | unmatched              | entity_id: 0.0.1.c1, guid_prefix: 1.f.ed.be.0.79.0.0.1.0.0.0
[1589905737.224691] trace    | FastEntities.cpp   | onParticipantDiscovery   | matched                | entity_id: 0.0.1.c1, guid_prefix: 1.f.ed.be.71.7a.0.0.1.0.0.0
[1589905743.449699] trace    | FastEntities.cpp   | onParticipantDiscovery   | unmatched              | entity_id: 0.0.1.c1, guid_prefix: 1.f.ed.be.71.7a.0.0.1.0.0.0

So we are a step further but I would say that it is currently not working as expected..?

pablogs9 commented 4 years ago

I use ros2 command line, try: ros2 service call [name] [type] "{a: 2, b: 3}"

Name and type should appear by using tab in the console.

gramss commented 4 years ago

yes this works for me:

$ ros2 service call /addtwoints example_interfaces/srv/AddTwoInts "{a: 2, b: 3}"
waiting for service to become available...
requester: making request: example_interfaces.srv.AddTwoInts_Request(a=2, b=3)

response:
example_interfaces.srv.AddTwoInts_Response(sum=5)

But the example_cpp node is still failing: https://github.com/ros2/demos/blob/c9e55acba6eaf377ccacce844853fd9eb20d1113/demo_nodes_cpp/src/services/add_two_ints_client.cpp#L79

Still an issue with the wait implementation..?

Edit: Same behavior with the eloquent version of the demo_node_cpp and the demo_node_py impl

Edit2: Even without the wait_for_service function the demo_node does not work as expected:

  // while (!client->wait_for_service(1s)) {
  //   if (!rclcpp::ok()) {
  //     RCLCPP_ERROR(node->get_logger(), "Interrupted while waiting for the service. Exiting.");
  //     return 0;
  //   }
  //   RCLCPP_INFO(node->get_logger(), "service not available, waiting again...");
  // }

  // TODO(wjwwood): make it like `client->send_request(node, request)->sum`
  // TODO(wjwwood): consider error condition
  auto result = send_request(node, client, request);
  if (result) {
    RCLCPP_INFO(node->get_logger(), "Result of add_two_ints: %zd", result->sum);
  } else {
    RCLCPP_ERROR(node->get_logger(), "Interrupted while waiting for response. Exiting.");
  }

This still gives me no result. Only when I crtl+C the program I get the last line as output.

pablogs9 commented 4 years ago

I'm not familiar with those cpp examples... sorry. Maybe QoS?

Take care with eloquent, the FastRTPS middleware in dashing (the one on the micro-ROS agent) may not be compatible with eloquent.

gramss commented 4 years ago

This is the python script run when we use ros2 service call... : https://github.com/ros2/ros2cli/blob/dashing/ros2service/ros2service/verb/call.py

Can you spot any difference to: https://github.com/ros2/demos/blob/dashing/demo_nodes_py/demo_nodes_py/services/add_two_ints_client.py

I'm not seeing any difference in the QoS - as they are not set..

I leave my finger from eloquent. I was just checking if it was a more striped version of the demo node..

gramss commented 4 years ago

the current implementation goes into a dead-lock or at least becomes unresponsive after a few fast requests: $ ros2 service call /addtwoints example_interfaces/srv/AddTwoInts "{a: 2, b: 3}" -r 2 you can easily trigger this with rates like 30 or higher. At 2 Hz it becomes unresonsive after like 30-40 calls.

I'm plaing with the sleep now. But besides not getting faster it should reliable answer my requests at its pace. Or am I missing something.. Maybe the request gets send during the sleeping phase and the client awaits a response while the service is waiting for the next one..?

edit: yes, I cannot reproduce this behavior when I comment out this line: https://github.com/micro-ROS/zephyr_apps/blob/d462926a89e54cef2199b97885e369655ae00bac/apps/add_two_ints_service/src/main.c#L66 So you should probably remove it. It can lead to race conditions that are really hard to recover from.. Maybe this can be detected somehow? If server ask for a new request but already got one -> resend last one..? Isn't that some sort of QoS?

edit2: nevermind.. also at rate 4000 with no ksleep this error occurs

pablogs9 commented 4 years ago

Have you tried to increase the history? https://github.com/micro-ROS/zephyr_apps/blob/d462926a89e54cef2199b97885e369655ae00bac/apps/add_two_ints_service/app-colcon.meta#L10

gramss commented 4 years ago

still happening with rate 4000 and History=10 after like 40 messages. I can restart my client on my host and the MCU happily answers them again. This supports my idea that the request gets lost in transition..

It even happens at rate = 1 and history= 10 after 10 requests...:

ros2 service call /addtwoints example_interfaces/srv/AddTwoInts "{a: 2, b: 3}" -r 1
waiting for service to become available...
requester: making request: example_interfaces.srv.AddTwoInts_Request(a=2, b=3)

response:
example_interfaces.srv.AddTwoInts_Response(sum=5)

requester: making request: example_interfaces.srv.AddTwoInts_Request(a=2, b=3)

response:
example_interfaces.srv.AddTwoInts_Response(sum=5)

requester: making request: example_interfaces.srv.AddTwoInts_Request(a=2, b=3)

response:
example_interfaces.srv.AddTwoInts_Response(sum=5)

requester: making request: example_interfaces.srv.AddTwoInts_Request(a=2, b=3)

response:
example_interfaces.srv.AddTwoInts_Response(sum=5)

requester: making request: example_interfaces.srv.AddTwoInts_Request(a=2, b=3)

response:
example_interfaces.srv.AddTwoInts_Response(sum=5)

requester: making request: example_interfaces.srv.AddTwoInts_Request(a=2, b=3)

response:
example_interfaces.srv.AddTwoInts_Response(sum=5)

requester: making request: example_interfaces.srv.AddTwoInts_Request(a=2, b=3)

response:
example_interfaces.srv.AddTwoInts_Response(sum=5)

requester: making request: example_interfaces.srv.AddTwoInts_Request(a=2, b=3)

response:
example_interfaces.srv.AddTwoInts_Response(sum=5)

requester: making request: example_interfaces.srv.AddTwoInts_Request(a=2, b=3)

response:
example_interfaces.srv.AddTwoInts_Response(sum=5)

requester: making request: example_interfaces.srv.AddTwoInts_Request(a=2, b=3)

^Csys:1: RuntimeWarning: Failed to fini client: rcl node implementation is invalid, at /tmp/binarydeb/ros-dashing-rcl-0.7.8/src/rcl/node.c:462

That makes not a reliable interface for me.. It would be therefor great if we can work out the error in order to produce a client on another host machine that works continuously. That way others can see that micro-ROS supports service flawlessly.

Generally speaking: benchmarking of different categories (pub/sub/service server/client /multiple nodes) would show others in a comprehensive overview what micro-ROS is currently capable of

pablogs9 commented 4 years ago

have you tested decreasing the wait set time? something like: rcl_wait(&wait_set,RCL_MS_TO_NS(1)); and setting the qos to : service_op.qos.reliability = RMW_QOS_POLICY_RELIABILITY_BEST_EFFORT;

gramss commented 4 years ago

tested it right now. It is still happening at various rates. I would even subjectively suggest that it is happening way faster. When I restart the client on my linux host it works again flawlessly until I hit ~the 10th request again.

I can even start another client in another bash, while the first one still hangs on waiting for a response from the service (running on the MCU). But the second one is still affected by the stop after a few reps.

BUT: I found the bug why the example demo_node client is not working.. Way to trivial.. The service must be called "add_two_ints" and not "addtwoints": https://github.com/micro-ROS/zephyr_apps/blob/46f1642f650261483a79456bdcdd46ab71454a85/apps/add_two_ints_service/src/main.c#L33

Maybe consider changing that as well in your rcl examples.

pablogs9 commented 4 years ago

Thanks for that discover, I will rename them when I have some time.

I have been testing Micro XRCE-DDS and it allows requesters and repliers at high rates without any problem, there must be some kind of error in the rmw. Let me some time to check it.

pablogs9 commented 4 years ago

Hey @gramss, we have found an (what seems to be) issue related to this lost services requests... it may take some days, but we are working on it.

gramss commented 4 years ago

Hey @pablogs9, thank you! I appreciate the update šŸ™‚ Keep me posted, if you solved the issue šŸ› Would be nice to see Micro-ROS getting a bit robuster after some "benchmarking" !

pablogs9 commented 4 years ago

Hey @gramss, sorry for the delay. I have good news, @julibert has found and fixed the bug and now we have this PR merged: https://github.com/eProsima/Micro-XRCE-DDS-Client/pull/156.

Please check it out using this new Micro XRCE-Client version. I have tested it in micro-ROS for Linux, and it works as expected at high rates. Could you check it on the embedded board and give us feedback?

gramss commented 4 years ago

Hey @pablogs9, thanks for getting back to me!

The error is sadly still there, at least I believe so. What I did was updating the Micro_XRCE-DDS-Client in the firmware folder under mcu_ws/eProsima. I did this by a git pull and switching to master (the script set this to dashing I believe..?)

And these are the changes I did in my code against your service example:

--- a/apps/add_two_ints_service/app-colcon.meta
+++ b/apps/add_two_ints_service/app-colcon.meta
@@ -7,7 +7,7 @@
                 "-DRMW_UXRCE_MAX_SUBSCRIPTIONS=0",
                 "-DRMW_UXRCE_MAX_SERVICES=1",
                 "-DRMW_UXRCE_MAX_CLIENTS=0",
-                "-DRMW_UXRCE_MAX_HISTORY=1"
+                "-DRMW_UXRCE_MAX_HISTORY=10"
             ]
         }
     }
diff --git a/apps/add_two_ints_service/src/main.c b/apps/add_two_ints_service/src/main.c
index 21404f7..4105d17 100644
--- a/apps/add_two_ints_service/src/main.c
+++ b/apps/add_two_ints_service/src/main.c
@@ -20,7 +20,7 @@ void main(void)

        // Optional RMW configuration 
        rmw_init_options_t* rmw_options = rcl_init_options_get_rmw_init_options(&options);
-       rmw_uros_options_set_serial_device("1", rmw_options);
+       rmw_uros_options_set_serial_device("2", rmw_options);
        RCCHECK(rmw_uros_options_set_client_key(0xDEADBEEF, rmw_options))

        rcl_context_t context = rcl_get_zero_initialized_context();
@@ -30,8 +30,9 @@ void main(void)
        rcl_node_t node = rcl_get_zero_initialized_node();
        RCCHECK(rcl_node_init(&node, "addtowints_server_rcl", "", &context, &node_ops))

-       const char * service_name = "addtwoints";
+       const char * service_name = "add_two_ints";
        rcl_service_options_t service_op = rcl_service_get_default_options();
+       service_op.qos.reliability = RMW_QOS_POLICY_RELIABILITY_BEST_EFFORT;
        rcl_service_t serv = rcl_get_zero_initialized_service();
        const rosidl_service_type_support_t * service_type_support = ROSIDL_GET_SRV_TYPE_SUPPORT(example_interfaces, srv, AddTwoInts);

@@ -46,7 +47,7 @@ void main(void)
                size_t index;
                RCSOFTCHECK(rcl_wait_set_add_service(&wait_set, &serv, &index))

-               rcl_wait(&wait_set, RCL_MS_TO_NS(100));
+               rcl_wait(&wait_set, RCL_MS_TO_NS(1));

                if (wait_set.services[index]) {   
                        rmw_request_id_t req_id;
@@ -63,7 +64,7 @@ void main(void)

                        RCSOFTCHECK(rcl_send_response(&serv,&req_id,&res))
                }
-               k_sleep(100);
+               //k_sleep(100);
        } while ( true );

After updating the Client, I did a build of the firmware and flashed it to my board.

I still can reproduce it with this:

$ ros2 service call /add_two_ints example_interfaces/srv/AddTwoInts "{a: 2, b: 3}" -r 30

Can you reproduce the error on your site with this?

pablogs9 commented 4 years ago

Hello @gramss, I have been testing and I have found out that:

It seems that the polling approach loss packets, so I have upgraded this serial transport with a rx async approach. Check the last commit of this branch.

With this async rx approach my board does not lose packages and the ros2 service call... does not freezes. It is slower than the USB approach or the UDP one, but this is expected as I'm using a 115200 bauds serial port.

Please test the new changes and give me feedback.

Thanks!

pablogs9 commented 4 years ago

For performance remember the service_op.qos.reliability = RMW_QOS_POLICY_RELIABILITY_BEST_EFFORT;

gramss commented 4 years ago

Hey @pablogs9 . Finally got back to my board. I just wanted to build it with the latest code but I ran into a new unrelated issue..?

Just created a fresh workspace and cloned the newest micro_ros_setup. Created my firmware folder Configured add_two_ints_server wanted to build and this happened:

/home/flo/uros_ws/firmware/zephyrproject/zephyr/include/net/buf.h:785:17: error: 'CONFIG_NET_BUF_USER_DATA_SIZE' undeclared here (not in a function)
  785 |  u8_t user_data[CONFIG_NET_BUF_USER_DATA_SIZE] __net_buf_align;

The code was already merged into dashing. So I wanted try to build it from scratch and test. The server_name is btw still not fixed.

pablogs9 commented 4 years ago

Please remind me what is do you refer with server_name.

I'm testing this error right now.

pablogs9 commented 4 years ago

Regaridng the error: this is now merged https://github.com/micro-ROS/zephyr_apps/pull/8 Error solved.

Please keep with your feedback, thanks!

gramss commented 4 years ago

Reminder :)

I found the bug why the example demo_node client is not working.. Way to trivial.. The service must be called "add_two_ints" and not "addtwoints":

https://github.com/micro-ROS/zephyr_apps/blob/46f1642f650261483a79456bdcdd46ab71454a85/apps/add_two_ints_service/src/main.c#L33

As the lines regarding setting the serial line are currently in the source of the service_example, I was able to test the changes pretty fast. It looked good for the moment. with rate 30Hz it run for a while. But it ultimately also came to an stop. I cannot restart the client. The board does not answer. Also restarting the agent has no affect. Only hard resetting the board is continuing to accept a new service request. (I did not play with any timings! Just changed the service_name and serial line to two) I used the same call like mentioned above: $ ros2 service call /add_two_ints example_interfaces/srv/AddTwoInts "{a: 2, b: 3}" -r 30 The hangup happened like a few minutes in. I was originally writing that everything seems to work now and just before I hit "comment" I rechecked and saw that it stopped.. I just retried it with 4000 Hz. Happened as well after a few minutes I would say. What is interesting is that the response time gets really slow just before the stop. You can see in the agent that it sends quite a few msg to the board until it finally receives an answer, or not. This is the -v6 output of the client when the board came to a sort of full stop:

[1591950880.984570] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 44 00 00 00 80
[1591950881.059557] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 44 00 44 00 80
[1591950881.086385] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 44 00 00 00 80
[1591950881.260331] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 44 00 44 00 80
[1591950881.289338] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 43 00 43 00 80
[1591950881.289556] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 44 00 00 00 80
[1591950881.390410] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 45 00 00 00 80
[1591950881.395239] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 44, data: 
0000: 81 80 44 00 07 01 24 00 00 4F 00 08 01 0F ED BE A3 D6 00 00 01 00 00 00 00 00 10 03 00 00 00 00
0020: 40 00 00 00 05 00 00 00 00 00 00 00
[1591950881.395541] debug    | Replier.cpp        | write                    | [** <<DDS>> **]        | client_key: 0x00000000, len: 32, data: 
0000: 01 0F ED BE A3 D6 00 00 01 00 00 00 00 00 10 03 00 00 00 00 40 00 00 00 05 00 00 00 00 00 00 00
[1591950881.395607] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 45 00 00 00 80
[1591950881.395993] debug    | SerialAgentLinux.cpp | recv_message             | [==>> SER <<==]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0A 01 05 00 45 00 00 00 80
[1591950881.396413] debug    | Replier.cpp        | read_fn                  | [==>> DDS <<==]        | client_key: 0x00000000, len: 40, data: 
0000: 01 0F ED BE A3 D6 00 00 01 00 00 00 00 00 10 03 00 00 00 00 41 00 00 00 02 00 00 00 00 00 00 00
0020: 03 00 00 00 00 00 00 00
[1591950881.396562] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 52, data: 
0000: 81 80 45 00 09 01 2C 00 00 0F 00 08 01 0F ED BE A3 D6 00 00 01 00 00 00 00 00 10 03 00 00 00 00
0020: 41 00 00 00 02 00 00 00 00 00 00 00 03 00 00 00 00 00 00 00
[1591950881.461995] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950881.662347] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950881.863215] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950882.063988] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950882.264578] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950882.465367] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950882.665925] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950882.866408] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950883.066856] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950883.268102] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950883.469000] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950883.669742] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950883.870145] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950884.071165] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950884.272177] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950884.472454] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950884.672671] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950884.873502] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950885.073950] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950885.274161] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950885.474882] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950885.675642] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950885.875990] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950886.076598] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950886.277618] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950886.478589] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950886.679758] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950886.879986] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950887.080237] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950887.280848] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950887.481330] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950887.682303] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950887.882449] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950888.083141] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950888.284844] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950888.485163] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950888.685548] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950888.886211] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950889.086718] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950889.287265] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950889.487782] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950889.688105] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950889.888716] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950890.089225] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950890.290072] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950890.490208] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950890.691466] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950890.891597] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950891.092598] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950891.293707] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950891.494324] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950891.694827] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950891.895726] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950892.096132] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950892.297106] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950892.497705] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950892.698062] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950892.898706] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950893.099560] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950893.300516] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950893.501517] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950893.701883] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950893.902224] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950894.102928] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950894.303290] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950894.504205] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950894.704878] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950894.905345] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950895.105930] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950895.306498] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950895.507235] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950895.707966] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950895.908706] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950896.108949] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950896.309412] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950896.510016] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950896.710714] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950896.910879] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950897.111868] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950897.312643] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950897.512983] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950897.713131] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950897.914158] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950898.114954] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950898.316022] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950898.517085] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950898.718022] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950898.918524] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950899.119872] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950899.320962] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950899.522019] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950899.722977] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950899.923213] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950900.124452] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950900.324826] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950900.525527] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950900.726201] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950900.926892] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950901.127643] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950901.328426] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950901.528970] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950901.730029] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950901.930910] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950902.131684] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950902.332373] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950902.533181] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950902.733623] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950902.934528] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950903.135236] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950903.335436] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950903.535810] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950903.736147] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950903.936442] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950904.137089] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950904.337807] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950904.538600] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950904.739280] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950904.940136] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950905.141763] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950905.342786] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950905.543523] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950905.744350] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950905.944992] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950906.145326] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950906.346369] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950906.546782] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950906.747500] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950906.948094] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950907.148795] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950907.349597] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950907.550061] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950907.750914] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950907.951161] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950908.151731] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950908.352081] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950908.553271] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950908.753967] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950908.954679] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950909.155388] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950909.356302] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950909.557069] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950909.757545] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950909.958736] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950910.159445] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950910.360136] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950910.561263] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950910.761551] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950910.962497] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950911.162778] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591950911.363351] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 13, data: 
0000: 81 00 00 00 0B 01 05 00 45 00 45 00 80
[1591951003.720529] trace    | FastEntities.cpp   | onSubscriptionMatched    | unmatched              | entity_id: 0.0.10.3, guid_prefix: 1.f.ed.be.a3.d6.0.0.1.0.0.0
[1591951003.720679] trace    | FastEntities.cpp   | onPublicationMatched     | unmatched              | entity_id: 0.0.f.4, guid_prefix: 1.f.ed.be.a3.d6.0.0.1.0.0.0
[1591951003.721570] trace    | FastEntities.cpp   | onParticipantDiscovery   | unmatched              | entity_id: 0.0.1.c1, guid_prefix: 1.f.ed.be.a3.d6.0.0.1.0.0.0
[1591951008.016845] trace    | FastEntities.cpp   | onParticipantDiscovery   | matched                | entity_id: 0.0.1.c1, guid_prefix: 1.f.ed.be.d9.d6.0.0.1.0.0.0
[1591951008.041539] trace    | FastEntities.cpp   | onPublicationMatched     | matched                | entity_id: 0.0.f.4, guid_prefix: 1.f.ed.be.d9.d6.0.0.1.0.0.0
[1591951008.122876] trace    | FastEntities.cpp   | onSubscriptionMatched    | matched                | entity_id: 0.0.10.3, guid_prefix: 1.f.ed.be.d9.d6.0.0.1.0.0.0
[1591951008.292912] debug    | Replier.cpp        | read_fn                  | [==>> DDS <<==]        | client_key: 0x00000000, len: 40, data: 
0000: 01 0F ED BE D9 D6 00 00 01 00 00 00 00 00 10 03 00 00 00 00 01 00 00 00 02 00 00 00 00 00 00 00
0020: 03 00 00 00 00 00 00 00
[1591951008.293051] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 52, data: 
0000: 81 80 46 00 09 01 2C 00 00 0F 00 08 01 0F ED BE D9 D6 00 00 01 00 00 00 00 00 10 03 00 00 00 00
0020: 01 00 00 00 02 00 00 00 00 00 00 00 03 00 00 00 00 00 00 00
[1591951012.967724] trace    | FastEntities.cpp   | onSubscriptionMatched    | unmatched              | entity_id: 0.0.10.3, guid_prefix: 1.f.ed.be.d9.d6.0.0.1.0.0.0
[1591951012.967783] trace    | FastEntities.cpp   | onPublicationMatched     | unmatched              | entity_id: 0.0.f.4, guid_prefix: 1.f.ed.be.d9.d6.0.0.1.0.0.0
[1591951012.968627] trace    | FastEntities.cpp   | onParticipantDiscovery   | unmatched              | entity_id: 0.0.1.c1, guid_prefix: 1.f.ed.be.d9.d6.0.0.1.0.0.0
[1591951016.422672] trace    | FastEntities.cpp   | onParticipantDiscovery   | matched                | entity_id: 0.0.1.c1, guid_prefix: 1.f.ed.be.e9.d6.0.0.1.0.0.0
[1591951016.447026] trace    | FastEntities.cpp   | onPublicationMatched     | matched                | entity_id: 0.0.f.4, guid_prefix: 1.f.ed.be.e9.d6.0.0.1.0.0.0
[1591951016.447099] trace    | FastEntities.cpp   | onSubscriptionMatched    | matched                | entity_id: 0.0.10.3, guid_prefix: 1.f.ed.be.e9.d6.0.0.1.0.0.0
[1591951016.699340] debug    | Replier.cpp        | read_fn                  | [==>> DDS <<==]        | client_key: 0x00000000, len: 40, data: 
0000: 01 0F ED BE E9 D6 00 00 01 00 00 00 00 00 10 03 00 00 00 00 01 00 00 00 02 00 00 00 00 00 00 00
0020: 03 00 00 00 00 00 00 00
[1591951016.699466] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 52, data: 
0000: 81 80 47 00 09 01 2C 00 00 0F 00 08 01 0F ED BE E9 D6 00 00 01 00 00 00 00 00 10 03 00 00 00 00
0020: 01 00 00 00 02 00 00 00 00 00 00 00 03 00 00 00 00 00 00 00
[1591951020.528123] trace    | FastEntities.cpp   | onSubscriptionMatched    | unmatched              | entity_id: 0.0.10.3, guid_prefix: 1.f.ed.be.e9.d6.0.0.1.0.0.0
[1591951020.528167] trace    | FastEntities.cpp   | onPublicationMatched     | unmatched              | entity_id: 0.0.f.4, guid_prefix: 1.f.ed.be.e9.d6.0.0.1.0.0.0
[1591951020.529093] trace    | FastEntities.cpp   | onParticipantDiscovery   | unmatched              | entity_id: 0.0.1.c1, guid_prefix: 1.f.ed.be.e9.d6.0.0.1.0.0.0

And this happens when I try to restart the client:

[1591951713.616623] trace    | FastEntities.cpp   | onSubscriptionMatched    | unmatched              | entity_id: 0.0.10.3, guid_prefix: 1.f.ed.be.e6.e1.0.0.1.0.0.0
[1591951713.616779] trace    | FastEntities.cpp   | onPublicationMatched     | unmatched              | entity_id: 0.0.f.4, guid_prefix: 1.f.ed.be.e6.e1.0.0.1.0.0.0
[1591951713.617132] trace    | FastEntities.cpp   | onParticipantDiscovery   | unmatched              | entity_id: 0.0.1.c1, guid_prefix: 1.f.ed.be.e6.e1.0.0.1.0.0.0
[1591951717.429180] trace    | FastEntities.cpp   | onParticipantDiscovery   | matched                | entity_id: 0.0.1.c1, guid_prefix: 1.f.ed.be.fb.e1.0.0.1.0.0.0
[1591951717.454683] trace    | FastEntities.cpp   | onPublicationMatched     | matched                | entity_id: 0.0.f.4, guid_prefix: 1.f.ed.be.fb.e1.0.0.1.0.0.0
[1591951717.535039] trace    | FastEntities.cpp   | onSubscriptionMatched    | matched                | entity_id: 0.0.10.3, guid_prefix: 1.f.ed.be.fb.e1.0.0.1.0.0.0
[1591951717.706980] debug    | Replier.cpp        | read_fn                  | [==>> DDS <<==]        | client_key: 0x00000000, len: 40, data: 
0000: 01 0F ED BE FB E1 00 00 01 00 00 00 00 00 10 03 00 00 00 00 01 00 00 00 02 00 00 00 00 00 00 00
0020: 03 00 00 00 00 00 00 00
[1591951717.707217] debug    | SerialAgentLinux.cpp | send_message             | [** <<SER>> **]        | client_key: 0xDEADBEEF, len: 52, data: 
0000: 81 80 48 00 09 01 2C 00 00 0F 00 08 01 0F ED BE FB E1 00 00 01 00 00 00 00 00 10 03 00 00 00 00
0020: 01 00 00 00 02 00 00 00 00 00 00 00 03 00 00 00 00 00 00 00
[1591951722.190711] trace    | FastEntities.cpp   | onSubscriptionMatched    | unmatched              | entity_id: 0.0.10.3, guid_prefix: 1.f.ed.be.fb.e1.0.0.1.0.0.0
[1591951722.190765] trace    | FastEntities.cpp   | onPublicationMatched     | unmatched              | entity_id: 0.0.f.4, guid_prefix: 1.f.ed.be.fb.e1.0.0.1.0.0.0
[1591951722.191683] trace    | FastEntities.cpp   | onParticipantDiscovery   | unmatched              | entity_id: 0.0.1.c1, guid_prefix: 1.f.ed.be.fb.e1.0.0.1.0.0.0

Does not answer for like a minute. Did not test this until ultimo if there is coming something.. What is better: the stop is now independend of the speed of the request (but I did not remove the wait inside the main..) What is worse: if the service stops to become unreachable you need to hard reset the board. This was "better" before as he recovered by restarting the client (not the agent!)

Offtopic: As soon as students want to test this next year with this board they would need examples that fit this board. Probably I or somebody from the laboratory will fork this repo (likely to be foxy then) and set the serial lines up accordingly.. :)

gramss commented 4 years ago

Interesting..

Removing this line here: https://github.com/micro-ROS/zephyr_apps/blob/46f1642f650261483a79456bdcdd46ab71454a85/apps/add_two_ints_service/src/main.c#L64

Brings back the old behavior. (answering some requests, failing, but a restart of the client gets you again the same number of requests like before)

Edit I can even reproduce this with 1Hz here..: ros2 service call /add_two_ints example_interfaces/srv/AddTwoInts "{a: 2, b: 3}" -r 1

pablogs9 commented 4 years ago

Regarding the fork, please make demos as generic as possible and PR them to this repo to have a great set of examples. Maybe if you generate some documentation for your course we can collaborate integrating it in our sites.

Regarding foxy, we are hard-working on it right now and I hope that it is ready very soon.

Finally, regarding the services issues, give some time to test it.

pablogs9 commented 4 years ago

@gramss you will have the correct name in foxy release ok?

https://github.com/micro-ROS/zephyr_apps/blob/8196c4e6e1e2dbb71375a10cd07f2341f8842ca8/apps/add_two_ints_service/src/main.c#L28

gramss commented 4 years ago

@pablogs9 Iā€˜m okay with that. Setup a 20.04 vm yesterday. Is this already a fix? I had no time right now to check into the code

pablogs9 commented 4 years ago

Not a fix but it will be changed in the Foxy release.

pablogs9 commented 4 years ago

Hey @gramss I have found that in Micro XRCE-DDS dashing branch is at v1.2.1 and the services fix is in v1.2.4. So I have upgraded the dashing branch to v1.2.4 in order to have the fix by default in the dashing XRCE middleware. Which branch were you using?

Please check it with a clean installation of the build system, and create a new firmware folder.

My test in the ST Disco IoT with the serial transport (using the STLink) has been working for at least 1 hour with:

ros2 service call /addtwoints example_interfaces/srv/AddTwoInts -r 10

When I have stopped it, it was already working. Please check this dashing issue and tell me your results.

pablogs9 commented 4 years ago

Hello @gramss how are you?

I was wondering if you have tested the new micro-ROS Foxy release. I was also wondering if we can close this issue.

Let me know what do you think.