ros-misc-utilities / ffmpeg_image_transport

ROS2 image transport plugin for encoding/decoding with h264 codec
Apache License 2.0
65 stars 22 forks source link

[HW acceleration on Jetson] #28

Closed PierreInnodura closed 6 months ago

PierreInnodura commented 6 months ago

Hello,

I try to republish a image topic with this package on a Jeston Orin AGX.

Everything works well in command line: image

But when trying to do the same with a launch file it fails and gives this output: image

My launch file: image

Everything works fine with libx264, but when trying to set hw accelerated encoder it fails.

Could you help me on this topic ?

Thank you.

berndpfrommer commented 6 months ago

Not sure what's going wrong here. You can try setting the parameters with something like this in your launch file:

parameters=[{
                    'ffmpeg_image_transport.encoding': 'hevc_nvenc',
                    'ffmpeg_image_transport.profile': 'main',
                    'ffmpeg_image_transport.preset': 'll',
                    'ffmpeg_image_transport.gop': 15,
                }]

but I doubt it'll make much of a difference. The parameters appear to be set correctly, the plugin is logging, so it's installed and found as well. For some reason when the node is started through the launch file, the ffmpeg library it's using there cannot find the hevc_nvenc encoder, whereas when you run from the command line, it can find it. I suspect it has to do with the difference in launch environments. Try this: use strace -f <your command here> >& /tmp/foo.txt to run from the command line and to run with the launch file. This will produce a huge amount of output since strace logs every system call. Now search the foo.txt file for anything related to ffmpeg_image_transport, libav and other interesting stuff. You may be able to see where it opens nvenc related stuff (successful run). You should also see what version of libavcodec (aka ffmpeg) is being used. Does it come from the same location? Compare the output from the two runs. What are the differences? Does libavcodec open the same files in both cases?

berndpfrommer commented 6 months ago

Oh, and please don't post screenshots. They are not easy to search, don't display well on e.g. my phone etc. Cut-and-paste the output and enclose with triple backquote. Thanks!

PierreInnodura commented 6 months ago

Thank you for your detailed, response but in fact my first mistake was to search for the nvenc codec but it's not available on Jetson platform. So I had to compile ffmpeg from source following this tutorial to add the Jetson twin of hevc_nvenc (hevc_nvmpi).

When configuring ffmpeg before compilation I had to enable shared libraries otherwise the compilation of this package wasn't working: ./configure --enable-nvmpi --disable-static --enable-shared

berndpfrommer commented 6 months ago

I have a xavier agx and got NVMPI working with a custom build ffmpeg library. But since you had an Orin and said it works when you run from the command line I figured they added nvenc support for Jetson. Apparently not so. So did you get everything to work? BTW IIRC there are some issues when encoding more than two images (stereo cam!). I think it segfaults on me because nvmpi could only run one stream.

PierreInnodura commented 6 months ago

So on my side I run two video stream (1920x1080 with I420 pixel format) with h264_nvmpi, both of them are running on the nvenc chipset which is running at full speed apparently. Everything seems to run perfectly so I'm very pleased of the result. Thank you for your amazing work !

PierreInnodura commented 6 months ago

It would be nice to add support to change the parameters of the nvmpi codec because its AVcontext seems different from the nvenc, I was only able to change the profile of the codec for now with this package, and I tried to change manually the default value of the codec before compilation since I'm not a C expert, but it didn't work unfortunately.

berndpfrommer commented 6 months ago

I'm very busy with other things at the moment, but please open a new issue and describe in more detail what you want to have supported, and I'll look at it. I should also document the NVMPI stuff for the jetson.

tonynajjar commented 3 months ago

I have a xavier agx and got NVMPI working with a custom build ffmpeg library. I should also document the NVMPI stuff for the jetson.

@berndpfrommer any chance you'll have some time soon to document it? I'd love to know how you did it. It's not exactly my setup but still will be useful

berndpfrommer commented 3 months ago

I have a xavier agx and got NVMPI working with a custom build ffmpeg library. I should also document the NVMPI stuff for the jetson.

@berndpfrommer any chance you'll have some time soon to document it? I'd love to know how you did it. It's not exactly my setup but still will be useful

Done