Can I use Image that belong sensor_msg?

[misumisumi]

Hello. Think you for your work. I'm use this repository for my own robot. Then, I have problem. I want to publish image from esp32-cam. But this library don't have sensor_msg::Image. There, I try creating custom msg, but this ros2 system different original ros2 system. Can you add sensor_msg::Image? or Will you teach how make custom msg? Think you.

[OpusK]

Hi, @Sumid1

ros2arduino uses eProsima's Micro-XRCE-DDS-Client. Therefore, the method of generating a message file is the same as that of eProsima's manual.

However, because ros2arduino has abstracted the API, the file created in this way should be abstracted from another API.

Here are some file links to reference:

• Topic abstraction API
• A message created by inheriting a Topic object and using a file created by the above generation.
• Must assign a unique ID for the message type.

I'll add and create sensor_msgs::Image message ASAP.

And we welcome contributors. :)

[OpusK]

@Sumid1

What size photo data do you want to transfer? Do you want additional RGB 24bit information? I've tested it and it's hard to send so much data.

Through the 'ROS2 Topic echo' command, the size of the data that was normally checked was about 32x32. If you want to send RGB 24bit information, you will only be able to send images that are 18x18 or smaller. Is this worth it to you?

And in addition, the default library state makes it difficult to transfer images properly, and you will have to make some settings yourself. (For example, setting communication buffer size)

First I'll put my work on another branch. Please test if necessary.

The user should add the information(data_size, data_array_capacity) not included in msg by (de)serialize method of Micro-XRCE-DDS, so please refer to the example below.

#include <ros2arduino.h>

#include <WiFi.h>
#include <WiFiUdp.h>
#define SSID       "YOUR_SSID"
#define SSID_PW    "YOUR_SSID_PASSWORD"
#define AGENT_IP   "AGENT_IP_ADDRESS"
#define AGENT_PORT 2019 //AGENT port number

#define PUBLISH_FREQUENCY 2 //hz
#define MSG_TYPE sensor_msgs::Image

void publishMsg(MSG_TYPE* msg, void* arg)
{
  (void)(arg);

  static int cnt = 0;
  static uint8_t data[32*32];

  sprintf(msg->header.frame_id, "sensor_msgs::Image");
  msg->data = data;
  msg->data_size = sizeof(data);
  msg->data_data_array_capacity = sizeof(data); //Required when subscribed.
}

class MessagePub : public ros2::Node
{
public:
  MessagePub()
  : Node("ros2arduino_udp_pub_node")
  {
    ros2::Publisher<MSG_TYPE>* publisher_ = this->createPublisher<MSG_TYPE>("arduino_writer");
    this->createWallFreq(PUBLISH_FREQUENCY, (ros2::CallbackFunc)publishMsg, nullptr, publisher_);
  }
};

WiFiUDP udp;

void setup() 
{
  WiFi.begin(SSID, SSID_PW);
  while(WiFi.status() != WL_CONNECTED);

  ros2::init(&udp, AGENT_IP, AGENT_PORT);
}

void loop() 
{
  static MessagePub MessageNode;

  ros2::spin(&MessageNode);
}

[misumisumi]

Hi, @OpusK . Thank you for the advice. I will use image for object recognition so I want to QVGA or more size of RGB 24bit. I know esp32-cam gets heat as using camera server.(This is cam camera web server example of esp32-cam.) Therefore, I understood your result. I will try your advice and my idea. In addition, I will report my result.

[misumisumi]

Hello. I report result of this task. As you said, esp32 can't send image more 32*32 size. However, I don't understand esp32-cam well, especially camera-driver. In the last few days, I search many information and look source codes. Therefore, I divided this problem into two. First, esp32 is low power as compared to Raspberry Pi that one board pc. However, I got the information that esp32 can send image using web socket. Therefore, I think esp32 have potential. Second, camera driver for esp32 is difficult for me. This reason is I'm beginner about c & c++ and low layer controller. So, I haven't understand how to operate yet. I want to try more, but It is in a hurry so I stop to search and think about this. I'm sorry. I Intend to try to send using websocket, receive host pc and transport Image msg in host pc. Finally, I'm so grateful for your quick work. Think you.

[misumisumi]

Hello. This library depend on micro-xrce-dds so I searched the repository and got one interesting issue. According to the issue, micro-xrce-dds can't send more 3000 byte. I tyied change config file as in the issue. Specifically, I changed 2048 to 5000 and 80000. In the result, esp32 return a lot of backtrace. Therefore, esp32 can't become camera node in the moment. Thank you for going out so far.

[misumisumi]

I'm sorry. I mistake. I read micro-xrce-dds doc now. According to the doc, we must do cmake after we change config file. When I have time, I'll try.

[misumisumi]

I have question. Can I send image sensor to change config and rebuild micro-xrce-dds-client ? I think I mistake to send raw image. However, I think I send jpeg image using CompressedImage. I try micro-xrce-dds-gen but I can't do it.(gradle build succes, but I can't run. ( not command)) In the first place, Do I mistake to send image using esp32?

[OpusK]

@Sumid1,

In the first place, Do I mistake to send image using esp32?

I think sending the image via esp32 is not a problem. However, a large buffer (memory) is required to process the image.

In addition, this condition must be met by both sending and receiving sides, and although fragmentation transmission methods exist, both must be supported by both sending and receiving.

In other words, even if you set the MicroXRCEDDS Client to send large amounts of data, it is not possible if the FastRTPS used by ROS2 cannot receive large amounts of data.

To me, it would be nice to leave e ros2arduino and ask eProsima how to send large amounts of data between MicroXRCEDDS and FastRTPS. It would be nice to ask them how to send a large size Image.

Of course, esp32's memory allowances are very different from PC's, so you'll have to reconsider, but first you need to make sure that MicroXRCE and FastRTPS are functionally capable of sending and receiving data.

[misumisumi]

Think you. As you send, I'll listen eProsima.