immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
52.44k stars 2.78k forks source link

Corrupt transcoded video with rkmpp decoder. #13709

Open kaaku3 opened 1 month ago

kaaku3 commented 1 month ago

The bug

X265 4K videos, transcoded down to 720p x265 using rkmmp hardware accelerated encoding and decoding are sometimes corrupted causing looped playback and strange colours and artifacts when played back. Video seem fine with HW decoding disabled. It does not happen every time and is inconsistent but happens often.

The OS that Immich Server is running on

Armbian Bookworm

Version of Immich Server

V1.118.2

Version of Immich Mobile App

V1.118.0

Platform with the issue

Your docker-compose.yml content

Same as default

Your .env content

Same as default

Reproduction steps

1. 2. 3. ...

Relevant log output

No response

Additional information

No response

kaaku3 commented 3 weeks ago

Hi, Are you sure this is fixed? You mentioned in #13785 that HW decoding is untested. The problem I reported only affects HW decoding.

mertalev commented 3 weeks ago

I tested this separately and I think it's caused by this issue. You're most likely seeing tone-mapping artifacts that will be fixed by setting the peak luminance explicitly. Since the linked PR does just this, it should fix the issue.

kaaku3 commented 3 weeks ago

Thankyou very much for getting back to me, I appreciate all the hard work.

kaaku3 commented 2 weeks ago

HI, Unfortunately I do still have the same problem on hardware decoded transcoded videos in v1.120. It has significantly improved but is still happening in low light videos

mertalev commented 2 weeks ago

Hmm, I guess peak=100 (1000 nits) is off for certain videos. Can you share the output of mediainfo <path> or ffprobe -show_streams <path> for an affected video? Maybe we can use a different number based on certain metadata.

mertalev commented 2 weeks ago

Also, are the videos experiencing issues in 1.120.0 the same videos that looked corrupt in earlier versions? Or do different videos look corrupt now?

kaaku3 commented 2 weeks ago

Media Info

General Complete name : PXL_20241105_062315977.mp4 Format : MPEG-4 Format profile : Base Media Codec ID : isom (isom/iso2/mp41) File size : 54.8 MiB Duration : 7 s 250 ms Overall bit rate : 63.4 Mb/s Frame rate : 59.859 FPS Encoded date : 2024-11-05 06:23:25 UTC Tagged date : 2024-11-05 06:23:25 UTC xyz : +35.6247+139.4298/ com.android.manufacturer : Google com.android.model : Pixel 6a

Video ID : 3 Format : HEVC Format/Info : High Efficiency Video Coding Format profile : Main@L5.1@High Codec ID : hvc1 Codec ID/Info : High Efficiency Video Coding Duration : 7 s 250 ms Bit rate : 63.1 Mb/s Width : 3 840 pixels Height : 2 160 pixels Display aspect ratio : 16:9 Rotation : 90° Frame rate mode : Variable Frame rate : 59.859 FPS Minimum frame rate : 59.367 FPS Maximum frame rate : 60.362 FPS Real frame rate : 60.000 FPS Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Bits/(Pixel*Frame) : 0.127 Stream size : 54.5 MiB (100%) Title : VideoHandle Language : English Encoded date : 2024-11-05 06:23:25 UTC Tagged date : 2024-11-05 06:23:25 UTC Color range : Full Color primaries : BT.709 Transfer characteristics : BT.709 Matrix coefficients : BT.709 Codec configuration box : hvcC

Audio ID : 2 Format : AAC LC Format/Info : Advanced Audio Codec Low Complexity Codec ID : mp4a-40-2 Duration : 7 s 243 ms Bit rate mode : Constant Bit rate : 192 kb/s Channel(s) : 2 channels Channel layout : L R Sampling rate : 48.0 kHz Frame rate : 46.875 FPS (1024 SPF) Compression mode : Lossy Stream size : 170 KiB (0%) Title : SoundHandle Language : English Encoded date : 2024-11-05 06:23:25 UTC Tagged date : 2024-11-05 06:23:25 UTC

Other Type : meta Duration : 7 s 250 ms Bit rate mode : Variable

mertalev commented 2 weeks ago

Hmm, it doesn't seem like this is an HDR video. Could you follow these steps:

  1. Change immich's log level to debug in the admin settings under Log Settings
  2. Run transcoding for this asset, making sure you have hardware decoding enabled
  3. In the terminal, run docker logs immich_server
  4. Find the latest log that shows an ffmpeg command (has /usr/bin/ffmpeg in it)

I'm curious to see what the specific command is to see where this is going wrong.

kaaku3 commented 2 weeks ago

I set log level to debug, but unfortunately i do not see ffmpeg anything. I tried verbose and the only thing I see that is remotely related in the log is: [Nest] 17 - 11/09/2024, 1:18:31 PM VERBOSE [Api:LoggingInterceptor~p0aolibi] {"assetIds":[],"name":"transcode-video"}

kaaku3 commented 2 weeks ago

Disregard that, the problem was the option "refresh encoded videos" is not re-encoding the video... maybe i misunderstood the meaning of that feature.

I made a new video in low light and uploaded it with hardware decoding on, the result was corruption towards the end of the file.

ffmpeg -n 10 /usr/bin/ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -afbc rga -noautorotate -i upload/library/Kirk/2024/2024-11-09/PXL_20241109_133516985.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:2 -map 0:1 -g 256 -tag:v hvc1 -v verbose -vf scale_rkrga=720:-2:format=nv12:afbc=1 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/c160480f-610d-42d9-8cea-829489f5a58e/f2/b0/f2b017d0-38f9-4331-8f71-d7c71e48b0c2.mp4

I then renamed the file and reuploaded it with hardware decoding off and the resulting file was fine.

ffmpeg -n 10 /usr/bin/ffmpeg -i upload/library/Kirk/2024/2024-11-09/reupload.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:2 -map 0:1 -g 256 -tag:v hvc1 -v verbose -vf scale=720:-2 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/c160480f-610d-42d9-8cea-829489f5a58e/8d/4e/8d4e5afc-9692-4169-b813-67ba68b96fdd.mp4

Media info for file

General Complete name : Kirk/2024/2024-11-09/reupload.mp4 Format : MPEG-4 Format profile : Base Media Codec ID : isom (isom/iso2/mp41) File size : 78.3 MiB Duration : 10 s 515 ms Overall bit rate : 62.5 Mb/s Frame rate : 59.536 FPS Movie name : Re-upload Encoded date : 2024-11-09 13:35:28 UTC Tagged date : 2024-11-09 13:35:28 UTC xyz : +35.6879+139.4576/ com.android.model : Pixel 7a com.android.manufacturer : Google

Video ID : 3 Format : HEVC Format/Info : High Efficiency Video Coding Format profile : Main@L6@Main Codec ID : hvc1 Codec ID/Info : High Efficiency Video Coding Duration : 10 s 515 ms Bit rate : 62.2 Mb/s Width : 3 840 pixels Height : 2 160 pixels Display aspect ratio : 16:9 Rotation : 90° Frame rate mode : Variable Frame rate : 59.536 FPS Minimum frame rate : 42.959 FPS Maximum frame rate : 96.774 FPS Real frame rate : 60.000 FPS Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Bits/(Pixel*Frame) : 0.126 Stream size : 78.0 MiB (100%) Language : English Encoded date : 2024-11-09 13:35:28 UTC Tagged date : 2024-11-09 13:35:28 UTC Color range : Full Color primaries : BT.709 Transfer characteristics : BT.709 Matrix coefficients : BT.709 Codec configuration box : hvcC

Audio ID : 2 Format : AAC LC Format/Info : Advanced Audio Codec Low Complexity Codec ID : mp4a-40-2 Duration : 10 s 505 ms Bit rate mode : Constant Bit rate : 192 kb/s Channel(s) : 2 channels Channel layout : L R Sampling rate : 48.0 kHz Frame rate : 46.875 FPS (1024 SPF) Compression mode : Lossy Stream size : 246 KiB (0%) Language : English Encoded date : 2024-11-09 13:35:28 UTC Tagged date : 2024-11-09 13:35:28 UTC

Other Type : meta Duration : 10 s 515 ms Bit rate mode : Variable

I think you are right they are not HDR, but with the latest update corruption only seems to happen in low light videos, where as before the update it seemed more random.

edit. Just ran a few tests and the artifacts and problems seem to happen when in low light but then a strong light source appears in the video, causing looped video, artifacts and strange colours.

mertalev commented 2 weeks ago

In the transcoding settings under Advanced, could you set the keyframe interval to 60? I wonder if that has an effect here.

kaaku3 commented 2 weeks ago

Wow yes, its seems fine with "Maximum keyframe interval" set to 60, thank you so much. Is it bad to keep it at 60 as opposed to auto?

mertalev commented 2 weeks ago

It will make the videos very slightly bigger, but otherwise no downside as far as I know.

kaaku3 commented 2 weeks ago

Ok, thankyou. keeping the file size down was the reason i'm transcoding at all, as my mobile data plan is slow sometimes.

Not sure if it's related but i just noticed a very very thin line at the top and bottom of the video that is not there when I use software decoding.

mertalev commented 2 weeks ago

The default in your case is 256, for reference. Can you try setting it to, say, 128? I'm thinking of changing the default for RKMPP to remedy this, but also trying to minimize the impact to file size.

mertalev commented 2 weeks ago

cc @nyanmisaka if you have any insight into what's causing this behavior

kaaku3 commented 2 weeks ago

After deleting and re-uploading the same to files one a dark video and one bright. with 60 still sometimes is causing an issue on both bright and dark video. 128 was very bad. Haven't seen a problem with it set to 32 after a few uploads. The line I mentioned only affects hardware decoded videos and only visible on the android mobile app, not on web.

kaaku3 commented 2 weeks ago

Since software decode is working fine with rkmpp hardware encode on its defaults, is there not any settings to play with on the hardware decode side? Just after some googling it seems not everyone is getting perfect results with ffmpeg and drm_prime, could that be the problem? The hardware decoder or ffmpegs compatibility with drm_prime? I don't really understand any of this but is there an alternative to drm_prime with rkmpp?

mertalev commented 2 weeks ago

A max of 32 for a 60 FPS video is a keyframe roughly every 0.5s. This is okay, but I think it's the lowest that's reasonable to use.

What's particularly weird is that this is an encoding setting - it shouldn't be fixed by using software decoding. It's a combination of the frames being outputted by hardware decoding and the keyframe interval that's causing this issue.

I think drm_prime is needed for good performance so filters can directly access the frames in the GPU without copying them.

kaaku3 commented 2 weeks ago

I'd like to help in anyway I can but I'm not very knowledgeable but i can follow instructions, do any of the devs have access to an rkmpp device?

kaaku3 commented 2 weeks ago

I hope it doesn't seem like i'm complaining about the issue too much, I understand you must be busy on other things. I love immich and i'm just grateful that rkmpp encode works at all, otherwise i'd have to buy a more power hungry device.

nyanmisaka commented 2 weeks ago

What’s the SoC model and rockchip linux kernel version? (uname -a)

Can u share a sample clip and the full ffmpeg command line?

I may have time to take a look tomorrow.

mertalev commented 2 weeks ago

No worries! It's great that you're reporting on this so we know about it and can fix it.

I think the best option would be to open an issue in jellyfin-ffmpeg as I believe most/all of the RKMPP decoders, filters and encoders for FFmpeg are implemented by them. A zipped sample video would be particularly helpful.

mertalev commented 2 weeks ago

According to OP, this command is problematic:

ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -afbc rga -noautorotate -i upload/library/Kirk/2024/2024-11-09/PXL_20241109_133516985.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:2 -map 0:1 -g 256 -tag:v hvc1 -v verbose -vf scale_rkrga=720:-2:format=nv12:afbc=1 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/c160480f-610d-42d9-8cea-829489f5a58e/f2/b0/f2b017d0-38f9-4331-8f71-d7c71e48b0c2.mp4

This command works fine:

ffmpeg -i upload/library/Kirk/2024/2024-11-09/reupload.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:2 -map 0:1 -g 256 -tag:v hvc1 -v verbose -vf scale=720:-2 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/c160480f-610d-42d9-8cea-829489f5a58e/8d/4e/8d4e5afc-9692-4169-b813-67ba68b96fdd.mp4

And changing the first command to use -g 32 instead of -g 256 also works fine.

Also, it seems there's a line at the top and bottom of the video when hardware decoding that doesn't happen with software decoding. I think this happens with -g 32 as well, but I'm not sure.

kaaku3 commented 2 weeks ago

uname -a Linux immich-server 5.10.160-legacy-rk35xx #1 SMP Wed May 15 03:04:45 UTC 2024 aarch64 GNU/Linux Its actually a rk3588s. This is my sample video of a messy (excuse the mess) dark room that constantly causes issues. 2 files one with -g256 and one with -g 32. In this instance -g 32 is also corrupt (up to this point -g32 was actually fine) It seems hit or miss.

Thankyou for the command line noted above and the description is perfect, the line thing happens no matter the -g setting and is only visible in mobile app (android) maybe web viewer clips it off?

w00tlarr commented 1 week ago

Hiya - running into this issue too but with Intel iGPU hardware acceleration. No issues encoding without iGPU.

w00tlarr commented 1 week ago

Sample video after Intel iGPU hardware transcode using VAAPI or QuickSync.

https://github.com/user-attachments/assets/2c0df285-4d35-440c-94cf-ba0dae5469d1

mertalev commented 1 week ago

Thanks for the report, but I don't think this is the same issue. The issue in question should be specific to RKMPP and (I think) fixed by this PR.

Please make a separate issue for this. You can set the log level to debug under Log Settings, then select Refresh encoded videos for an affected asset. Find the FFmpeg command used for this video in the server logs with docker logs immich_server. In the issue, provide a sample video that causes this output and the accompanying FFmpeg command. Also include information about the OS, kernel version and specific processor.

kaaku3 commented 1 week ago

Great to hear a fix is on the way, thankyou so much. (I was beginning to wonder if my chip was faulty)

kaaku3 commented 1 week ago

So the reason it didn't happen with software decode was because it was slower?

nyanmisaka commented 1 week ago

Sample video after Intel iGPU hardware transcode using VAAPI or QuickSync.

playback.mp4

This is a completely different issue. Due to the driver/kernel rather than FFmpeg.

https://github.com/intel/media-driver/issues/1665

nyanmisaka commented 1 week ago

Great to hear a fix is on the way, thankyou so much. (I was beginning to wonder if my chip was faulty)

The version in the link above should fix this artifact. You can try to install it manually and try again.

As for the green line I couldn't reproduce it in Chrome, PotPlayer, MPC-BE and even FFplay. So it is a problem specific to the built-in video player of your client platform.

720x406 hevc main 720x406

kaaku3 commented 1 week ago

Thankyou, I'm not nearly versed enough with docker to update ffmpeg from source,I will wait for this to reach immich in an update. The line, I was only about to reproduce in the immich app and only with files hardware decided and encoded.

You can see it here top left, and bottom middle in this screenshot Screenshot_20241111-175451

nyanmisaka commented 1 week ago

Thankyou, I'm not nearly versed enough with docker to update ffmpeg from source,I will wait for this to reach immich in an update. The line, I was only about to reproduce in the immich app and only with files hardware decided and encoded.

You can see it here top left, and bottom middle in this screenshot

It's best to switch to a different client and try again.

nyanmisaka commented 1 week ago

For example, I cannot reproduce it on iPhone/iOS 17.

IMG_8534

kaaku3 commented 1 week ago

Ok, I will try it with the updated immich client, once the fix reaches it maybe it's not an issue anymore. If it's still an issue I will open up an issue with the immich mobile app for android.

w00tlarr commented 1 week ago

Sample video after Intel iGPU hardware transcode using VAAPI or QuickSync. playback.mp4

This is a completely different issue. Due to the driver/kernel rather than FFmpeg.

intel/media-driver#1665

Thanks. Yeah it's that bug with the Intel Media Driver link you provided. Not sure if I should open a separate bug now as it seems to be a very very old issue with Proxmox + SRIOV + my new iPhone 16 with HDR videos. I just noticed this issue because of my new iPhone is recording HDR videos vs. my old iPhone XR.

Disabling the Tone-Mapping in the Immich Video Transcoding setting now works properly with hardware acceleration.

Anyways, sorry for hijacking this thread as it read like the same issue with me. I can open a new bug thread for tips to other folks, if needed.

nyanmisaka commented 1 week ago

Sample video after Intel iGPU hardware transcode using VAAPI or QuickSync. playback.mp4

This is a completely different issue. Due to the driver/kernel rather than FFmpeg. intel/media-driver#1665

Thanks. Yeah it's that bug with the Intel Media Driver link you provided. Not sure if I should open a separate bug now as it seems to be a very very old issue with Proxmox + SRIOV + my new iPhone 16 with HDR videos. I just noticed this issue because of my new iPhone is recording HDR videos vs. my old iPhone XR.

Disabling the Tone-Mapping in the Immich Video Transcoding setting now works properly with hardware acceleration.

Anyways, sorry for hijacking this thread as it read like the same issue with me. I can open a new bug thread for tips to other folks, if needed.

I don't have an Intel GPU that supports SRIOV at the moment (Intel ARC A380), you could try looking in the Intel repo to provide some context. But anyway this issue is not related to the downstream software, as the same FFmpeg commands run perfectly on the host.

mertalev commented 1 week ago

@kaaku3 The bug fix for the RKMPP issue should be in 1.120.2. Can you try with that?

kaaku3 commented 1 week ago

@mertalev that's great news, unfortunately my armbian server that immich was running went down last night. I didn't touch anything, I don't know why. You think publicly putting out a link to a video on immich was a bad idea? I'm sure people have better things to do then mess with my server probably over thinking it. Anyway I think I will need to put the immich backup restore feature to use today. I will let you know if it's fixed as soon as i can. Thankyou fire your help.

mertalev commented 1 week ago

Sorry to hear that! There are lots of reasons it could fail, like power loss. But in general, make sure the server is hardened with a reverse proxy, fail2ban, etc. if you're exposing it to the internet. Definitely don't expose Immich (or any other service) directly.

kaaku3 commented 1 week ago

I was using nginx, I will look into fail2ban too thankyou.

kaaku3 commented 1 week ago

Hi, I started fresh. I had a lot of problems seems the newer kernel don't fully support hardware acceleration... i'm not quite sure. I went with Joshua Rieks ubuntu with legacy kernel, that apparent everything should work out of the box, but i had problems with opencl. It seems to be working now, but I'm not 100%. Is there anyway from immich log to tell if its falling back to software encode. It doesn't seem to be going as fast as it was before. Also immich restore didn't work for me so i'm uploading everything.

this is the ffmpeg from immich log

ffmpeg -n 10 /usr/bin/ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -afbc rga -noautorotate -i upload/upload/0175ee76-7502-46c4-9685-08b06e26c9dc/a7/f0/a7f09b0f-653d-4920-8a95-ec929b890cc8.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:2 -map 0:1 -g 256 -tag:v hvc1 -v verbose -vf scale_rkrga=720:-2:format=nv12:afbc=1 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/0175ee76-7502-46c4-9685-08b06e26c9dc/f9/57/f9578312-e623-4a61-9667-607baed5a02e.mp4

anyway to tell from that that it isn't falling back to software? I'm currently waiting on the problem file to be transcoded.

edit.

I there are errors

[Nest] 7 - 11/14/2024, 1:30:47 AM ERROR [Microservices:MediaRepository] handler_name : SoundHandle vendor_id : [0][0][0][0] Stream #0:20x3: Video: hevc (Main), 1 reference frame (hvc1 / 0x31637668), yuvj420p(pc, bt709, left), 3840x2160, 62337 kb/s, SAR 1:1 DAR 16:9, 59.86 fps, 59.94 tbr, 90k tbn (default) Metadata: creation_time : 2024-09-27T11:15:49.000000Z handler_name : VideoHandle vendor_id : [0][0][0][0] [out#0/mp4 @ 0x3b53a170fc0] Adding streams from explicit maps... [vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Created video stream from input stream 0:2 [hevc_rkmpp @ 0x3b53a2d1d80] Found SoC name from device-tree: 'rockchip,rk3588s-orangepi-5 rockchip,rk3588' [hevc_rkmpp @ 0x3b53a2d1d80] Picked up an existing RKMPP hardware device [aost#0:1/aac @ 0x3b53a1f0780] Created audio stream from input stream 0:1 Stream mapping: Stream #0:2 -> #0:0 (hevc (hevc_rkmpp) -> hevc (hevc_rkmpp)) Stream #0:1 -> #0:1 (aac (native) -> aac (native)) [vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Starting thread... [aost#0:1/aac @ 0x3b53a1f0780] Starting thread... [vf#0:0 @ 0x3b53a127d40] Starting thread... [af#0:1 @ 0x3b53a127e80] Starting thread... [vist#0:2/hevc @ 0x3b53a050d80] [dec:hevc_rkmpp @ 0x3b53a2357c0] Starting thread... [aist#0:1/aac @ 0x3b53a050c00] [dec:aac @ 0x3b53a237200] Starting thread... [in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Starting thread... Press [q] to stop, [?] for help [hevc_rkmpp @ 0x3b53a2d1d80] Noticed an info change [hevc_rkmpp @ 0x3b53a2d1d80] Configured with size: 3840x2160 | pix_fmt: drm_prime | sw_pix_fmt: nv12 [graph 0 input from stream 0:2 @ 0x3b542090300] w:3840 h:2160 pixfmt:drm_prime tb:1/90000 fr:60000/1001 sar:1/1 csp:bt709 range:pc [Parsed_scale_rkrga_0 @ 0x3b542090240] w:3840 h:2160 fmt:nv12 -> w:1280 h:720 fmt:nv12 [graph 0 input from stream 0:2 @ 0x3b542090300] video frame properties congruent with link at pts_time: 0 [hevc_rkmpp @ 0x3b53a2d0500] Using input frames context (format drm_prime) with hevc_rkmpp encoder. [hevc_rkmpp @ 0x3b53a2d0500] Rate Control mode is set to CQP [hevc_rkmpp @ 0x3b53a2d0500] QP Init/Max/Min/Max_I/Min_I is set to 28/28/28/28/28 [hevc_rkmpp @ 0x3b53a2d0500] Tier is set to 1 [hevc_rkmpp @ 0x3b53a2d0500] Profile is set to MAIN [hevc_rkmpp @ 0x3b53a2d0500] Level is set to 51 [hevc_rkmpp @ 0x3b53a2d0500] Configured with size: 1280x720 | pix_fmt: drm_prime | sw_pix_fmt: nv12 [graph_1_in_0:1 @ 0x3b540090300] tb:1/48000 samplefmt:fltp samplerate:48000 chlayout:stereo Output #0, mp4, to 'upload/encoded-video/0175ee76-7502-46c4-9685-08b06e26c9dc/d1/6d/d16d540c-1124-4d90-847d-53fb08e97a41.mp4': Metadata: major_brand : isom minor_version : 131072 compatible_brands: isomiso2mp41 com.android.capture.fps: 60.000000 location : +35.6813+139.5357/ location-eng : +35.6813+139.5357/ com.android.manufacturer: Google com.android.model: Pixel 6a encoder : Lavf61.1.100 Stream #0:0(eng): Video: hevc (Main), 1 reference frame (hvc1 / 0x31637668), drm_prime(pc, bt709, progressive, left), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 2000 kb/s, 59.94 fps, 60k tbn (default) Metadata: creation_time : 2024-09-27T11:15:49.000000Z handler_name : VideoHandle vendor_id : [0][0][0][0] encoder : Lavc61.3.100 hevc_rkmpp Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, delay 1024, 128 kb/s (default) Metadata: creation_time : 2024-09-27T11:15:49.000000Z handler_name : SoundHandle vendor_id : [0][0][0][0] encoder : Lavc61.3.100 aac [out#0/mp4 @ 0x3b53a170fc0] Starting thread... frame= 138 fps=0.0 q=-0.0 size= 1024KiB time=00:00:02.02 bitrate=4133.1kbits/s speed=4.06x [hevc_rkmpp @ 0x3b53a2d0500] Failed to get key input frame from packet meta: -1 [vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Error submitting video frame to the encoder [vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Error encoding a frame: Generic error in an external library [vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Task finished with error code: -542398533 (Generic error in an external library) [vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Terminating thread with return code -542398533 (Generic error in an external library) [vf#0:0 @ 0x3b53a127d40] All consumers returned EOF [vf#0:0 @ 0x3b53a127d40] Terminating thread with return code 0 (success) [vist#0:2/hevc @ 0x3b53a050d80] [dec:hevc_rkmpp @ 0x3b53a2357c0] Decoder returned EOF, finishing [vist#0:2/hevc @ 0x3b53a050d80] [dec:hevc_rkmpp @ 0x3b53a2357c0] Terminating thread with return code 0 (success) [vist#0:2/hevc @ 0x3b53a050d80] All consumers of this stream are done [in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Terminating thread with return code 0 (success) [aist#0:1/aac @ 0x3b53a050c00] [dec:aac @ 0x3b53a237200] Decoder thread received EOF packet [aist#0:1/aac @ 0x3b53a050c00] [dec:aac @ 0x3b53a237200] Decoder returned EOF, finishing [aist#0:1/aac @ 0x3b53a050c00] [dec:aac @ 0x3b53a237200] Terminating thread with return code 0 (success) [af#0:1 @ 0x3b53a127e80] Filtergraph returned EOF, finishing [af#0:1 @ 0x3b53a127e80] All consumers returned EOF [aost#0:1/aac @ 0x3b53a1f0780] Encoder thread received EOF [aost#0:1/aac @ 0x3b53a1f0780] Terminating thread with return code 0 (success) [out#0/mp4 @ 0x3b53a170fc0] All streams finished [out#0/mp4 @ 0x3b53a170fc0] Terminating thread with return code 0 (success) [af#0:1 @ 0x3b53a127e80] Terminating thread with return code 0 (success) [mp4 @ 0x3b53a190e00] Starting second pass: moving the moov atom to the beginning of the file [AVIOContext @ 0x3b53a237fc0] Statistics: 2787923 bytes read, 0 seeks [AVIOContext @ 0x3b53a237c00] Statistics: 5582216 bytes written, 4 seeks, 25 writeouts [out#0/mp4 @ 0x3b53a170fc0] Output file #0 (upload/encoded-video/0175ee76-7502-46c4-9685-08b06e26c9dc/d1/6d/d16d540c-1124-4d90-847d-53fb08e97a41.mp4): [out#0/mp4 @ 0x3b53a170fc0] Output stream #0:0 (video): 283 frames encoded; 278 packets muxed (2720898 bytes); [out#0/mp4 @ 0x3b53a170fc0] Output stream #0:1 (audio): 192 frames encoded (196608 samples); 193 packets muxed (66981 bytes); [out#0/mp4 @ 0x3b53a170fc0] Total: 471 packets (2787879 bytes) muxed [out#0/mp4 @ 0x3b53a170fc0] video:2657KiB audio:65KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 0.230928% frame= 278 fps=272 q=-0.0 Lsize= 2729KiB time=00:00:04.12 bitrate=5425.4kbits/s speed=4.04x [aac @ 0x3b53a2d2480] Qavg: 832.837 [in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Input file #0 (upload/upload/0175ee76-7502-46c4-9685-08b06e26c9dc/40/f9/40f9b5f3-8512-4058-9ffe-b71c12358bac.mp4): [in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Input stream #0:1 (audio): 193 packets read (99038 bytes); 192 frames decoded; 0 decode errors (196608 samples); [in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Input stream #0:2 (video): 303 packets read (39917376 bytes); 289 frames decoded; 0 decode errors; [in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Total: 496 packets (40016414 bytes) demuxed [AVIOContext @ 0x3b53a230180] Statistics: 40437714 bytes read, 4 seeks Conversion failed!

I don't think i can test this right now

mertalev commented 1 week ago

It seems like this is the relevant line:

[hevc_rkmpp @ 0x3b53a2d0500] Failed to get key input frame from packet meta: -1

Does it fail for all videos, or only certain ones?

kaaku3 commented 1 week ago

Thankyou for pointing that out,it seems to be only one file. I was starting to think something was up with the drivers. Looks as tho it was uploaded via an immich client on my wife's phone, is it possible it was something up with the upload... It fell back to software transcode and passed that ok

mertalev commented 1 week ago

Thanks for clarifying. I'm assuming the corruption issue is resolved now?

I'm not sure what the issue is for that particular video, but if it worked with software decoding then it's most likely a bug.

kaaku3 commented 1 week ago

Yes, I just transcoded that file and it's fixed, thankyou so much. There is the line still but maybe that's a different issue. I got a few more errors like the last I will start again with my system make sure there is nothing up with the GPU drivers. You wouldn't have any recommendations on a base image for orange pi 5 would you?

mertalev commented 1 week ago

ubuntu-rockchip tends to be recommended as the best for hardware acceleration support.

kaaku3 commented 1 week ago

Ok, thankyou

mertalev commented 1 week ago

You can also try downgrading back to 1.120.1 to see if you still get the same error. That would make it clear whether the issue is due to the different server environment or the FFmpeg change.