Closed akssri-sony closed 1 year ago
Could you please post the output (first few lines) of 'ros2 echo /your_image_topic'? I'd like to see what encoding is being used by the encoder?
The problem is apparently that the decoder tries to use some cuda decoder. Not sure why. Could you please also post the line with which you launch the republisher (decoder)?
The command for starting the republisher in the README almost certainly will have to be modified to avoid using the accelerated decoder. Maybe something like this:
ros2 run image_transport republish ffmpeg in/ffmpeg:=image_raw/ffmpeg raw out:=image_raw/uncompressed --ros-args -p "ffmpeg_image_transport.map.hevc_nvenc:=libx264"
This is what I get when I echo the topic with ros2 topic echo /camera/color/image_rect_raw/ffmpeg
header:
stamp:
sec: 1682568513
nanosec: 971701760
frame_id: camera_color_optical_frame
width: 848
height: 480
encoding: libx264
pts: 1394
flags: 0
is_bigendian: false
data:
- 0
- 0
- 0
- 1
- 65
- 155
- 211
- 120
- 67
- 252
- 153
- 76
- 0
- 130
- 31
- 130
- 15
- 106
- 70
- 132
- 92
- 54
- 11
- 5
- 182
- 224
- 235
- 183
- 147
- 56
- 48
- 149
- 103
-
The republisher was initially run with ros2 run image_transport republish ffmpeg in/ffmpeg:=/camera/color/image_rect_raw/ffmpeg raw out:=/camera/color/image_rect_raw/uncompressed
.
I tried setting all the mappings with
ros2 run image_transport republish ffmpeg in/ffmpeg:=/camera/color/image_rect_raw/ffmpeg raw out:=/camera/color/image_rect_raw/uncompressed --ros-args -p "ffmpeg_image_transport.map.h264_nvmpi:=libx264" --ros-args -p "ffmpeg_image_transport.map.h264_nvenc:=libx264" --ros-args -p "ffmpeg_image_transport.map.hevc_nvenc:=libx264" --ros-args -p "ffmpeg_image_transport.map.libx264:=libx264"
Only the last one had any effect (since the stream is encoded with libx264). Now I get,
root@garuda:/home/ros/humble# ros2 run image_transport republish ffmpeg in/ffmpeg:=/camera/color/image_rect_raw/ffmpeg raw out:=/camera/color/image_rect_raw/uncompressed --ros-args -p "ffmpeg_image_transport.map.h264_nvmpi:=libx264" --ros-args -p "ffmpeg_image_transport.map.h264_nvenc:=libx264" --ros-args -p "ffmpeg_image_transport.map.hevc_nvenc:=libx264" --ros-args -p "ffmpeg_image_transport.map.libx264:=libx264"
[WARN] [1682571049.574895085] [rcl]: Found remap rule 'in/ffmpeg:=/camera/color/image_rect_raw/ffmpeg'. This syntax is deprecated. Use '--ros-args --remap in/ffmpeg:=/camera/color/image_rect_raw/ffmpeg' instead.
[WARN] [1682571049.574909802] [rcl]: Found remap rule 'out:=/camera/color/image_rect_raw/uncompressed'. This syntax is deprecated. Use '--ros-args --remap out:=/camera/color/image_rect_raw/uncompressed' instead.
[ERROR] [1682571049.637535289] [image_republisher]: SubscriberPlugin::subscribeImpl with five arguments has not been overridden
[ERROR] [1682571050.100268191] [FFMPEGDecoder]: cannot open decoder libx264
[ERROR] [1682571050.100311730] [FFMPEGDecoder]: cannot open decoder libx264
[ERROR] [1682571050.100349746] [FFMPEGSubscriber]: cannot initialize decoder!
[ERROR] [1682571051.100484505] [FFMPEGDecoder]: cannot open decoder libx264
[ERROR] [1682571051.100518457] [FFMPEGDecoder]: cannot open decoder libx264
[ERROR] [1682571051.100523935] [FFMPEGSubscriber]: cannot initialize decoder!
Some digging on the internet led me to something that says that ffmpeg needs to be recompiled with spec. flags; not sure if this is the issue.
root@garuda:/home/ros/humble# ffmpeg -codecs | grep -i 264
ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil 56. 70.100 / 56. 70.100
libavcodec 58.134.100 / 58.134.100
libavformat 58. 76.100 / 58. 76.100
libavdevice 58. 13.100 / 58. 13.100
libavfilter 7.110.100 / 7.110.100
libswscale 5. 9.100 / 5. 9.100
libswresample 3. 9.100 / 3. 9.100
libpostproc 55. 9.100 / 55. 9.100
DEV.LS h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_v4l2m2m h264_qsv h264_cuvid ) (encoders: libx264 libx264rgb h264_nvenc h264_omx h264_qsv h264_v4l2m2m h264_vaapi nvenc nvenc_h264 )
All the decoders for h264 have some CUDA error associated with them.
Looks like get_hw_type
in ffmpeg_decoder.cpp always returns the AV_HWDEVICE_TYPE_CUDA
(=2), regardless of whether there's a cuda device or not. Hacking this to take the non-Cuda code path leads to other errors: the decoded image is all messed up. Thanks for the help!
Input:
Output:
What happens if you run the republisher like this:
ros2 run image_transport republish ffmpeg in/ffmpeg:=image_raw/ffmpeg raw out:=image_raw/uncompressed --ros-args -p "ffmpeg_image_transport.map.libx264:=h264"
If you use straight ffmpeg (the executable) from the command line, can you encode/decode with libx264? If that works, then the ROS2 transport plugins should also work.
Oh, and your image resolution is really odd. The line size of 848 is not even a multiple of 32. Other users have reported weirdly decoded images in such situations. I'm surprised you didn't get a warning on the encoder side:
if ((width % 32) != 0) {
RCLCPP_WARN(logger_, "horiz res must be multiple of 32!");
}
If I run the republisher with,
ros2 run image_transport republish ffmpeg in/ffmpeg:=image_raw/ffmpeg raw out:=image_raw/uncompressed --ros-args -p "ffmpeg_image_transport.map.libx264:=h264"
I end up with the same error as in the first comment.
On the resolution side, I did get the warning, but didn't realize this would be the effect. Is there an easy fix for this. I guess padding would work, but is there a better way on the libav side ?
I really don't know how to fix this on the libav side. But does standalone ffmpeg work correctly for decoding h264 packets?
On Thu, Apr 27, 2023, 9:55 PM akssri-sony @.***> wrote:
If I run the republisher with,
ros2 run image_transport republish ffmpeg in/ffmpeg:=image_raw/ffmpeg raw out:=image_raw/uncompressed --ros-args -p "ffmpeg_image_transport.map.libx264:=h264"
I end up with the same error as in the first comment.
On the resolution side, I did get the warning, but didn't realize this would be the effect. Is there an easy fix for this. I guess padding would work, but is there a better way on the libav side ?
— Reply to this email directly, view it on GitHub https://github.com/berndpfrommer/ffmpeg_image_transport/issues/3#issuecomment-1526868969, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPLK2S7XONDDSV7K3CRRUTXDMPQPANCNFSM6AAAAAAXMBDPHM . You are receiving this because you commented.Message ID: @.***>
Fixed by PR #4
I run into the following ffmpeg decoder error on the sub-side under this setup,
Would be grateful if you have some insight into what's going wrong or how this could solved. Thanks!