[BUG] Bitrate of previews is way smaler than defined in ffmpeg settings.

[akoyaxd]

The bug

I tested the HWE feature for video Previews. I currently have the issue that the videos use way less bitrate than defined in max Bitrate.

My 720p Videos seem to only get about 900 kbit/s per second, which looks quite bad. Max Bitrate is set to 4500k.

I also tried to increase the Quality by setting the CRF Value to 1, without any affect. On 21 the issue is the same.

The OS that Immich Server is running on

Debian

Version of Immich Server

v1.72.1

Version of Immich Mobile App

v1.72.0

Platform with the issue

• [X] Server
• [ ] Web
• [ ] Mobile

Your docker-compose.yml content

on demand

Your .env content

on demand

Reproduction steps

1. Enable HWE in settings
2. Upload Video
3. Preview is low quality caused by low bitrate.

Additional information

System

i3-12100 with integrated graphics. Immich Docker container runs in lxc container in Proxmox 7.4.

Microservice log entry.

Start encoding video 16a1d654-7880-4583-9ff7-65cc76b00a72 {"inputOptions":["-init_hw_device qsv=hw","-filter_hw_device hw"],"outputOptions":["-vcodec h264_qsv","-g 256","-extbrc 1","-refs 5","-bf 7","-acodec aac","-movflags faststart","-fps_mode passthrough","-v verbose","-vf format=nv12,hwupload=extra_hw_frames=64,scale_qsv=-1:720","-preset 5","-threads 1","-global_quality 1","-maxrate 4500k","-bufsize 9000k"],"twoPass":false}

FFmpeg Settings

[mertalev]

Hi! The flags for these settings are definitely passed, so it's odd that it's not respecting them. Can you try with VAAPI as well? Also check the microservices logs to see if it's throwing any errors.

[akoyaxd]

Just tested VAAPI.

Preview Filesize is now 2900k to 3100k with max bitrate configured to 4500k. So denfinately an improvement.

[mertalev]

So it's local to QSV.

As for the bitrate you're getting with VAAPI, I double-checked the VAAPI bitrate setting and VAAPI doesn't support setting both a quantizer and a max bitrate. If max bitrate is set, it uses a lower target bitrate derived from the max bitrate and ignores CRF. If you want 4500k to be your target, you can set the max bitrate to 6500k. If you want the CRF to be used, you'll need to set the max bitrate to 0.

[Deathproof76]

The resulting ffmpeg command seems to be wrong for qsv. I'm having the exact opposite problem with bloated files

I translated these settings for qsv vp9 as an example:

ffmpeg -init_hw_device qsv=hw -filter_hw_device hw -i input.mp4 -vcodec vp9_qsv -g 256 -extbrc 1 -refs 5 -bf 7 -low_power 1 -acodec aac -movflags faststart -fps_mode passthrough -v verbose -vf format=nv12,hwupload=extra_hw_frames=64 -preset 3 -threads 14 -q:v 24 -maxrate 3200 -bufsize 6400 output.mp4 and the files are completely bloated with bitrates like 75+mbit/s

I removed-q:v and added -b:v:0 2250k -maxrate:v:0 3200k ffmpeg -init_hw_device qsv=hw -filter_hw_device hw -i input.mp4 -vcodec vp9_qsv -g 256 -extbrc 1 -refs 5 -bf 7 -low_power 1 -acodec aac -movflags faststart -fps_mode passthrough -v verbose -vf format=nv12,hwupload=extra_hw_frames=64 -preset 3 -threads 14 -b:v:0 2250k -maxrate:v:0 2250k -bufsize:v:0 3600k output.mp4

which produces files in the expected bitrate range

btw: switching on TWO-PASS ENCODING, with otherwise same settings in the webui, seems to be ignored. The resulting files are extremely bloated too

docker logs also confirm this: [Nest] 1939 - 08/12/2023, 2:05:50 AM LOG [MediaService] Start encoding video 3089cf23-4ed0-484b-92c0-ffd65c025f1b {"inputOptions":["-init_hw_device qsv=hw","-filter_hw_device hw"],"outputOptions":["-vcodec vp9_qsv","-g 256","-extbrc 1","-refs 5","-bf 7","-low_power 1","-acodec aac","-movflags faststart","-fps_mode passthrough","-v verbose","-vf format=nv12,hwupload=extra_hw_frames=64","-preset 3","-threads 14","-q:v 24","-maxrate 3200","-bufsize 6400"],"twoPass":false}

[mertalev]

The resulting ffmpeg command seems to be wrong for qsv. I'm having the exact opposite problem with bloated files

I translated these settings for qsv vp9 as an example:

ffmpeg -init_hw_device qsv=hw -filter_hw_device hw -i input.mp4 -vcodec vp9_qsv -g 256 -extbrc 1 -refs 5 -bf 7 -low_power 1 -acodec aac -movflags faststart -fps_mode passthrough -v verbose -vf format=nv12,hwupload=extra_hw_frames=64 -preset 3 -threads 14 -q:v 24 -maxrate 3200 -bufsize 6400 output.mp4 and the files are completely bloated with bitrates like 75+mbit/s

I removed-q:v and added -b:v:0 2250k -maxrate:v:0 3200k ffmpeg -init_hw_device qsv=hw -filter_hw_device hw -i input.mp4 -vcodec vp9_qsv -g 256 -extbrc 1 -refs 5 -bf 7 -low_power 1 -acodec aac -movflags faststart -fps_mode passthrough -v verbose -vf format=nv12,hwupload=extra_hw_frames=64 -preset 3 -threads 14 -b:v:0 2250k -maxrate:v:0 2250k -bufsize:v:0 3600k output.mp4

which produces files in the expected bitrate range

btw: switching on TWO-PASS ENCODING, with otherwise same settings in the webui, seems to be ignored. The resulting files are extremely bloated too

docker logs also confirm this: [Nest] 1939 - 08/12/2023, 2:05:50 AM LOG [MediaService] Start encoding video 3089cf23-4ed0-484b-92c0-ffd65c025f1b {"inputOptions":["-init_hw_device qsv=hw","-filter_hw_device hw"],"outputOptions":["-vcodec vp9_qsv","-g 256","-extbrc 1","-refs 5","-bf 7","-low_power 1","-acodec aac","-movflags faststart","-fps_mode passthrough","-v verbose","-vf format=nv12,hwupload=extra_hw_frames=64","-preset 3","-threads 14","-q:v 24","-maxrate 3200","-bufsize 6400"],"twoPass":false}

The issue is that low power mode is required for VP9, but this mode limits rate control options. When I tested VP9 for QSV, using -global_quality resulted in it using ICQ rate control and erroring out as it wasn't supported, so VP9 is handled with CQP (-q:v) mode instead. I can retest to see if I can get it working with ICQ, but otherwise it's kept this way for compatibility reasons.

For two-pass mode, only NVENC supports this (with the -multipass 2 flag), while the other APIs do not as far as I'm aware. This is mentioned in the docs, but it would be good to mention it in the setting description too.

[mertalev]

Also, the max bitrate field should have a k at the end to signify that it's in kbps. Omitting it should mean bytes per second, but I'm not sure if QSV assumes differently given the file sizes you're talking about. If it assumes a very large max bitrate instead, I can see -q:v 24 with a 4K video (given your target resolution is set to original) producing videos of that size.

[Deathproof76]

@mertalev sorry for the late reply Using qsv again, this time with a crf of 47, max bitrate at 7850k (added the "k", which shouldn't matter in this scenario anyways). Still, videos with a FHD original resolution get a preview that has at least 20 mbps. I also used different crf values like 27, 37 etc... the file is transcoded with the new setting (according to logs), the old preview is deleted but the resulting filesize of the new file remains the same:

used settings in full

[Deathproof76]

It sure would be nice to have something like an expert mode, where you could see the resulting ffmpeg cmd and also modify it.

[mertalev]

@Deathproof76 I'll see if I can reproduce this. Out of curiosity, have you tried VAAPI with VP9? Are the results between the two any different?

[Deathproof76]

@mertalev VAAPI and VP9 is working as far as I can tell. If set a max Bitrate of 9000K the resulting file settles at around 6000ish.

But even at slow the quality seems to be lacking, lots of blocky artefacting (comparable to h265 with about 1300k maybe). CPU-only encoding of Vp9 with a max bitrate of 7000k fares a lot better but still looks unusually blocky.

A UHD770 should do a lot better quality wise, but I don't have much experience with VAAPi and vp9, only qsv, nvenc and hevc. Will have to test outside of immich for myself to understand what I can expect. Right now it seems kind of broken.

[mertalev]

I did some testing with VAAPI on a 13th gen Intel machine that got some very strange results.

When CQP mode is used, normal values for -global_quality produce massive files as if the quantization parameter were set near 0. I had to increase this number all the way to 130 (!) for it to produce a similar bitrate to HEVC with -global_quality set to 23 (and otherwise equal settings).

Setting the rate control to ICQ instead completely ignored -global_quality, -q:v and -qp:v and produced potato quality videos. This is likely what the OP is describing since QSV will use ICQ unless coerced into CQP.

I didn't test VBR, but assume this one would be normal.

I'm not sure what to do with this since it seems like a bug either in the driver or in FFmpeg. I could try to map the user-inputted CRF to this weird scaling, but I'm not sure why this behaves the way it does or if this behavior is universal.

[mertalev]

I think this issue is fixed since CQP is now exposed as an option and defaults to being enabled if using VP9.