search

strukturag / libheif

libheif is an HEIF and AVIF file format decoder and encoder.
Other
1.72k stars 298 forks source link

Use hevc_cuvid instead of software decoding if available #1322

Open Neoclassic opened 6 days ago

Neoclassic commented 6 days ago

Feature request to use hardware decoding if available for hevc formats.

Please apprise if it already possible. I already have a ffmpeg dist with nvidia hevc decoder.

Although that works fine for some heic to png. But In case of images with tiles it fails to provide full image area as png.

bradh commented 6 days ago

You can try with the branch if you like: https://github.com/strukturag/libheif/pull/1296

However it should be possible to build the main branch against ffmpeg dist and use that. Its not enabled by default but you can compile it to be so.

The bigger issue is "what is happening when it fails" for your samples. Do you have a specific example that consistently shows the problem? Can you share that sample?

Neoclassic commented 5 days ago

@bradh

Case 1: Fails completely https://github.com/Neoclassic/heic_test_cases/blob/main/exif.heic

testfiles % ffprobe -hide_banner -i $loc                           
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test_folder_conversion_png_1/test/exif.heic':
  Metadata:
    major_brand     : mif1
    minor_version   : 0
    compatible_brands: mif1
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0[0x4e22]: Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv), 1280x720, 1 fps, 1 tbr, 1 tbn (default)
      Metadata:
        title           : HEVC Image
 testfiles % 
testfiles % ffmpeg -hide_banner -loglevel error -i $loc -map 0 %d.png
[hevc @ 0x13d805500] Invalid NAL unit size (0 > 176).
[hevc @ 0x13d805500] Error splitting the input into NAL units.
[vist#0:0/hevc @ 0x13c605f70] [dec:hevc @ 0x13d804900] Decoding error: Invalid data found when processing input
[vist#0:0/hevc @ 0x13c605f70] [dec:hevc @ 0x13d804900] Decode error rate 1 exceeds maximum 0.666667
[vist#0:0/hevc @ 0x13c605f70] [dec:hevc @ 0x13d804900] Task finished with error code: -1145393733 (Error number -1145393733 occurred)
[vist#0:0/hevc @ 0x13c605f70] [dec:hevc @ 0x13d804900] Terminating thread with return code -1145393733 (Error number -1145393733 occurred)

Case 2: https://github.com/Neoclassic/heic_test_cases/blob/main/LiveOff.HEIC

Type is : hevc (Main Still Picture)

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'LiveOff.HEIC':
  Metadata:
    major_brand     : heic
    minor_version   : 0
    compatible_brands: mif1heic
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream group #0:0[0x31]: Tile Grid: hevc (Main Still Picture) (hvc1 / 0x31637668), yuvj420p(pc), 4032x3024 (default)
  Stream #0:48[0x32]: Video: hevc (Main Still Picture) (hvc1 / 0x31637668), yuvj420p(pc), 320x240, 1 fps, 1 tbr, 1 tbn
      Side data:
        ICC Profile

So it has an embedded thumbnail as well Stream #0:48[0x32] of size 320x240, but requirement is convert full image to png.

One option is to get tiles and then montage them.

bradh commented 5 days ago

Have you tested with libheif and the ffmpeg decoder plugin?

Neoclassic commented 5 days ago

@bradh Yes, surely going to try. There is some learning curve for me here.

bradh commented 5 days ago

I decoded them OK with libheif (not HW accelerated though). Also, the iphone image has personal information in it, which you might want to strip out.

Neoclassic commented 5 days ago

Thanks bradh, I cannot work with software decoding due to legal challenges related with the format. But will try with your PR.

Neoclassic commented 1 day ago

@bradh

I built libheif using https://github.com/bradh/libheif/tree/nvdev_merge_2

But getting this error

/usr/local/bin/heif-dec /heifcases/LiveOff.HEIC out.png

[istream] request_range 0 - 1024 [istream] request_range 24 - 3946 [istream] request_range 15119 - 17157 File contains 1 image [istream] request_range 24447 - 46783 [istream] request_range 17157 - 24447 [istream] request_range 46783 - 78025 [istream] request_range 78025 - 103598 [hevc_cuvid @ 0x7220a0024640] Invalid pkt_timebase, passing timestamps as-is. [hevc_cuvid @ 0x7220a801d0c0] Invalid pkt_timebase, passing timestamps as-is. [hevc_cuvid @ 0x722094025f80] Invalid pkt_timebase, passing timestamps as-is. [hevc_cuvid @ 0x72209c028bc0] Invalid pkt_timebase, passing timestamps as-is. Could not decode image: 0: Decoder plugin generated an error: Unspecified: avcodec_receive_frame returned EAGAIN or ERROR_EOF

Neoclassic commented 1 day ago

Same problem with main branch as well

cmake .. -DCMAKE_INSTALL_RPATH=/libheifbin -DWITH_DAV1D=OFF -DWITH_GDK_PIXBUF=OFF -DWITH_RAV1E=OFF -DWITH_SvtEnc=OFF -DWITH_FFMPEG_DECODER=ON