Store in rosbag and beeing able to replay

[nullket]

Hello,

to be able to deal with my available data rate I was very happy to stumble across this project! Using noetic your code works perfectly for live feeds. Well done!

But I am not able to store the images/video in a rosbag. Have you experience with that? My test was:

• record just /camera/image_raw/ffmpeg in a bag
• replay that bag (on a fresh new roscore with nothing running)
• open rqt_image_view and try to subscribe to the image - but here the whole mission fails as I am not able to see the topic in the drop down menu. I can perfectly see the topic beeing published using rostopic hz /camera/image_raw/ffmpeg

[berndpfrommer]

It's definitely possible to replay from a bag, but it's been a while since I've done that. I think the problem is that the rosbag player does not instantiate an image transport so you need to actually tell the client what transport you want to use, in this case "ffmpeg". Have a look at the answer here. https://answers.ros.org/question/129266/how-to-i-rosbagrecord-a-loosless-compressed-images/

My guess would be that in your case this would become [THIS IS WRONG, SEE BELOW FOR RIGHT ANSWER]:

rosrun image_view image_view image:=/camera/image_raw ffmpeg

Or maybe use as image "/camera/image_raw/ffmpeg". The important part is the last keyword which selects the transport. Also make sure that before you run the image_view, in that terminal the ffmpeg image transport plugin is visible (there's a way to list all available transport plugins in ros). Otherwise image_view cannot decode the ffmpeg packets.

[nullket]

Thanks for the quick answer! I had an urgent presentation so it took me a bit until I could check your proposal and play around with the plugin:

I was able to to record it in a bag as one would expect:

rosback record /camera/image_raw/ffmpeg

and replay it with:

rosrun image_view image_view image:=/camera/image_raw  _image_transport:=ffmpeg

So to sum it up: it is perfectly possible to store the topic in bags and replay them. Nonetheless, rqt_image_view does not work out of the box (even I specify _image_transport:=ffmpeg). Actually, that was the reason why I opened this issue, I initially just tried it with rqt_image_view. If you do not mind we could leave this issue open and I will check within the next weeks whats the matter with rqt_image_view and report back.

[berndpfrommer]

will leave open, thanks for testing!

[nullket]

I have done some further tests

Summing it up

• Assuming a live feed rqt_image_view works out of the box
• Saving the transport specific topic works out of the box
• Replaying a rosbag works as well except rqt_image_view (see below)
• rqt_image_view seems to ignore settings like _image_transport on the command line (which make image_view work)
• rqt_image_view does not list the ffmpeg topic when replaying from a bag. The problem is how rqt_image_view retrieves the available topics. It looks through all topics and checks if their message type is sensor_msgs/Image. If yes it is simply assumed that this topic is the standard image_raw topic and it adds it and also suffixes all available transport plugins and adds those as well. Thus rqt_image_view works perfectly fine with ffmpeg for live feeds when the image_raw topic is available. But in rosbags this particular base topic is missing. To cover this for the default compressed plugin the maintainers added second check with the so called message_sub_types variable. So all one needs to do is to add message_sub_types.insert("ffmpeg_image_transport_msgs/FFMPEGPacket"); in the source code as well.
• Unfortunately, this can not be done automatically or more flexible as image_transports API only exposes the plugins name but not its message type which makes it very hard to determine whether a topic is a image topic or not.
• I am not trying to submit an official fix/extension for rqt_image_view as this plugin is not in the official ros repos anyways.

TL:DR All works but it needs a local extension of rqt_image_view

[berndpfrommer]

Thanks for digging deep on this issue and the nice writeup! At some point I may rewrite this plugin for ROS2 and try to get (the ROS2 version of) the plugin into the official ROS repos. The dependency on a custom built ffmpeg would then have to be removed.