search

Keylost / jetson-ffmpeg

ffmpeg support on nvidia jetson
Other
78 stars 26 forks source link

Add resize option to nvmpi decoder #8

Closed madsciencetist closed 1 year ago

madsciencetist commented 1 year ago

This mirrors the -resize option that the cuvid decoders support. The resize operation happens for free while converting from BlockLinear to PitchLinear.

madsciencetist commented 1 year ago

Oops, this isn't actually working yet.

Keylost commented 1 year ago

Have you tested this with the ffmpeg CLI tool? This will not work as expected because the output frame will not have the correct dimensions. ff_get_buffer uses the sizes specified in AVCodecContext.

ffmpeg -c:v h264_nvmpi -resize:v 1920x1080 -i 235_short.mkv -c: h264 235_t.mkv Input #0, matroska,webm, from '235_short.mkv': Metadata: title : Media Presentation ENCODER : Lavf60.3.100 Duration: 00:01:07.20, start: 0.000000, bitrate: 14175 kb/s Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 3840x2160, 18 fps, 18 tbr, 1k tbn, 36 tbc (default) Output #0, matroska, to '235_t.mkv': Metadata: title : Media Presentation encoder : Lavf58.29.100 Stream #0:0: Video: h264 (h264_nvmpi) (H264 / 0x34363248), yuv420p, 3840x2160, q=-1--1, 2000 kb/s, 18 fps, 1k tbn, 18 tbc (default)

madsciencetist commented 1 year ago

I found that issue, and I think I fixed it in the commit I just pushed, but I'm still seeing the following behavior:

I see this both with and without WITH_NVUTILS defined.

Testable with ffplay -vcodec hevc_nvmpi -resize WIDTHxHEIGHT -i INPUT

Keylost commented 1 year ago

I thought I fixed it, but I think I was too hasty... downscaling 3840x2160->1920x1080 looks fine now. upscaling 1920x1080->3840x2160 still not working... any ideas?

Keylost commented 1 year ago

....too hasty :( downscaling also doesn't work correctly....

Keylost commented 1 year ago

p.s. All changes still in master branch. I decided not to revert them for now.

Keylost commented 1 year ago

Fixed. The problem was that the ffmpeg buffer had a different alignment than the frame in video memory. NvBuffer2Raw/NvBufSurface2Raw doesn't seem to be able to properly handle such cases, so I replaced them. This problem could occur even without scaling at non-standard resolutions for which the line data size would be different from the size of the line in memory in the ffmpeg buffer. It's great that we found this issue) @madsciencetist, thank you for your contribution!

madsciencetist commented 1 year ago

Oh, nice job @Keylost! I was way behind, still trying to get NvEglRenderer to work so I could just visualize the dma buffer. I'll test right now.

Testing your code now, I'm getting a segfault...let me see what that's about.

madsciencetist commented 1 year ago

With WITH_NVUTILS, no segfault. Without WITH_NVUTILS, I get a segfault either in NvBufferTransform(), memcpy(dataDst, dataSrc, copySz), or in tegra lib worker threads.

Segfault occurs even without resizing

Keylost commented 1 year ago

Fixed. Try latest commit. @madsciencetist

madsciencetist commented 1 year ago

That worked! Thank you!

madsciencetist commented 1 year ago

@Keylost last request - before anybody pulls this, think you could you could clean up master by squashing all these resize commits together? I image others will want to pull in this feature to their forks, so it would be great if there was just a single commit to be cherrypicked.