deserialize SerializedMessage into arbitrary message type on runtime?

[nyxaria]

Hello,

I am just starting to use ROS2 and am looking into creating a generic subscriber which can subscribe to any topic, get rclcpp::SerializedMessage and then convert it to the message_type of the topic that the message came from.

So far I have the generic subscriber working and have a SerializedMessage, but am confused how to go about converting it to a normal ros message.

My thought is to use the Serialization.deserialize_message. To do this, I would need a function which does "std_msgs::msg::String" -> std_msgs::msg::String (ie string name of message_type -> actual message_type class). Is there something which achieves this?

What I'm ultimately trying to achieve is to get a key-value vector of all the data inside any message.

Furthermore, for this approach I would need to create a Serialization object for each message_type. Is there a better way?

Many thanks

[Karsten1987]

There's pretty much exactly what you need inside rosbag2_transport:

• https://github.com/ros2/rosbag2/blob/master/rosbag2_transport/src/rosbag2_transport/generic_publisher.hpp
• https://github.com/ros2/rosbag2/blob/master/rosbag2_transport/src/rosbag2_transport/generic_subscription.hpp

There was work started in here to factor this out of rosbag and make it available within the C++ library.

[nyxaria]

Thank you @Karsten1987, I have used a generic subscriber to get SerializedMessages, but I'm unsure how to extract the data from it into a key value pair array.

[nyxaria]

I have found this code from here

    my_pkg::msg::Msg msg;
    auto ros_message = std::make_shared<rosbag2_introspection_message_t>();
    ros_message->time_stamp = 0;
    ros_message->message = nullptr;
    ros_message->allocator = rcutils_get_default_allocator();
    ros_message->message = &msg;
    auto type_support = rosbag2::get_typesupport("my_pkg/msg/Msg", "rosidl_typesupport_cpp");

    rosbag2::SerializationFormatConverterFactory factory;
    std::unique_ptr<rosbag2::converter_interfaces::SerializationFormatDeserializer> cdr_deserializer_;
    cdr_deserializer_ = factory.load_deserializer("cdr");

    cdr_deserializer_->deserialize(msg, type_support, ros_message);

Is there some way to make the type of msg set from the message_type string (ie "my_pkg/msg/Msg")?

Maybe I'm stuck in a XY problem, I'm not even sure if this is needed and if I can somehow extract the key value data directly from the SerializedMessage. What I basically need to achieve is generic subscriber -> JSON representation of any message

[Karsten1987]

does the snippet from the demos help you here? It's converting an incoming serialized message into a ros2 message. https://github.com/ros2/demos/blob/master/demo_nodes_cpp/src/topics/listener_serialized_message.cpp#L56-L63

I feel though your ticket here is actually better suited for a question at https://answers.ros.org/questions/

[nyxaria]

I asked on ros answers, but did not get anywhere. The demo you posted is ALMOST what I want -- in that code, you know the data type is going to be std_msgs::msg::String, but what if you only had the message type as a string from node.get_topic_names_and_types()? Can you change this using MessageT = std_msgs::msg::String; in this demo into using MessageT = class_from_string("std_msgs::msg::String")?

Thanks @Karsten1987 for your help, I'm new to ROS2 and I'm just trying to read the source code to figure this stuff out but there is a lot of ground to cover.

[Karsten1987]

The rclcpp::Serialization class takes a pointer to a message type support: https://github.com/ros2/rclcpp/blob/master/rclcpp/include/rclcpp/serialization.hpp#L56

So the question really boils down to how to get this typesupport from a string. That's a little bit tricky, but can be done with the typesupport helper functions we've implemented within rosbag2: https://github.com/ros2/rosbag2/blob/master/rosbag2_cpp/include/rosbag2_cpp/typesupport_helpers.hpp Example use case: https://github.com/ros2/rosbag2/blob/master/rosbag2_cpp/test/rosbag2_cpp/test_typesupport_helpers.cpp#L79-L82

[clalancette]

@nyxaria @Karsten1987 Can we close this issue now?

[nyxaria]

Yes, thank you for your help

[agoeckner]

Hello @nyxaria. Would you be able to post your completed code?

Also, is there a way to do this using ROS2 Eloquent? It seems that Eloquent does not have the rclcpp::Serialization class mentioned above by @Karsten1987 .

[Karsten1987]

The functionality was already available in Eloquent, unfortunately just not in a clean C++ API: https://github.com/ros2/demos/blob/eloquent/demo_nodes_cpp/src/topics/talker_serialized_message.cpp#L58-L112

[agoeckner]

Thanks for the response, @Karsten1987! That's very helpful. My issue right now is that I don't know the message type at compile-time. I only have a string describing the type, like "sensor_msgs/msg/BatteryState", from which I derive the rosidl_message_type_support_t object. How can I create a message object given only the rosidl_message_type_support_t?

[Karsten1987]

That should be answered via the previous comment in here. It's part of the typesupport helpers within rosbag2, where we load the message type based on a string. It's essentially a dynamic lookup of the library where we expect the symbol for the rosidl_message_type_support_t to be present.

[Uhrendoktor]

For any one else looking into this in the future wanting a concrete example. I've written something along the lines in a recent project, where I edit frame_ids in Headers of arbitrary message at runtime: https://github.com/Uhrendoktor/tf2_remapper/tree/main

[xhy279]

For any one else looking into this in the future wanting a concrete example. I've written something along the lines in a recent project, where I edit frame_ids in Headers of arbitrary message at runtime: Uhrendoktor/tf2_remapper@main

Just for clear, I can parse message with its string type and without its actualy type, but I still have to know the actual message type in order to use it, right? As you wrote here: (https://github.com/Uhrendoktor/tf2_remapper/blob/c8b15a174e558bdf2fe44e607e77e5ffd9ea30b4/src/Tf2Remapper.cpp#L282C60-L282C95)

[Uhrendoktor]

For any one else looking into this in the future wanting a concrete example. I've written something along the lines in a recent project, where I edit frame_ids in Headers of arbitrary message at runtime: Uhrendoktor/tf2_remapper@main

Just for clear, I can parse message with its string type and without its actualy type, but I still have to know the actual message type in order to use it, right? As you wrote here: (https://github.com/Uhrendoktor/tf2_remapper/blob/c8b15a174e558bdf2fe44e607e77e5ffd9ea30b4/src/Tf2Remapper.cpp#L282C60-L282C95)

Not quite sure what you mean by "using" it. If you want to use the Cpp type of course you have to get the type. But you can just deserialize -> modify -> serialize without knowing the type (basic example to access fields without knowing the type. https://github.com/Uhrendoktor/tf2_remapper/blob/main/include/tf2_remapper/DynLibCpp.hpp#L151, where MessageMember and MessageMembers hold the following data https://github.com/ros2/rosidl/blob/15609e2ee204cfbb2b024de3552b9c806e152111/rosidl_typesupport_introspection_cpp/include/rosidl_typesupport_introspection_cpp/message_introspection.hpp). Or dersialize only part of the message: in my example only the header, ignoring the rest. An alternativ would be to turn the entire message into a format like json or yaml (https://github.com/osrf/dynamic_message_introspection) or something along the lines, if you'd want to make a nice UI for editting. If for your usecase you somehow really need the type you could also dynamically load it at runtime. Not too sure what to do with that though.