Add HDR (or Dolby Vision) color-mapping options

[DigitalWriting-Dev]

Since 10.8.0, Jellyfin is capable of color mapping of Dolby Vision content. Scrubbing thumbnails of Dolby Vision content has weird colors. Adding an option to enable color-mapping would really help with that kind of content, since the thumbnails look pretty out of place. An example of a Dolby Vision thumbnail.

[nicknsy]

Hi, I don't have any hdr compatible things I couldn't actually test it but the newest release might fix this. Lmk what happens if you try regenerating it

[DigitalWriting-Dev]

I'm still generating things at 8% after 20-ish hours. Hopefully changing the priority will help, but in any case how do I force regeneration for that specific movie to test this?

Edit: Nevermind, manually found the trickplay directory and removed it. Can't check the thumbnails yet for some reason but will update. Regarding the speed, it's way faster now.

Edit 2: So, after deleting the trickplay directories I can't create any bif files due to lack of permissions. Tried chmodding and chowning the config directory but that didn't help.

Edit 3: Looks like the permissions error is a v1.0.0.3 issue, thumbnails are being generated with 1.0.0.2. I did a few tests and looks like the priority setting is what's causing the permissions error. Since I'm creating all of the thumbnails from scratch it'll take me a long time to be able to test the color mapping bug again.

[nicknsy]

Could you paste the error you get? Also, what's your environment? Docker, installed from apt, etc. If docker, do you have UID/PID/GUID tags in your compose file to run as a specific user?

[nicknsy]

Also if you have on demand generation on you can click on the dolby vision video and it should start generation. Although I guess you can't test it if 1.0.0.3 isnt working at all

[DigitalWriting-Dev]

Could you paste the error you get? Also, what's your environment? Docker, installed from apt, etc. If docker, do you have UID/PID/GUID tags in your compose file to run as a specific user?

Here's the log from one of the files when using the highest priority:

jellyfin    | [14:01:39] [INF] [82] Nick.Plugin.Jellyscrub.Drawing.OldMediaEncoder: /usr/lib/jellyfin-ffmpeg/ffmpeg -f matroska,webm -i file:"/homevideos/קלטות/tape 7  03-04-1999---25-02-2000.mkv" -threads 0 -v quiet -filter:v fps=1/10,scale=min(iw\,320):trunc(ow/dar/2)*2 -f image2 "/config/cache/temp/6ac286a0336244b1a40dd8617b23ee07/img_%08d.jpg"
jellyfin    | [14:01:39] [INF] [30] Emby.Server.Implementations.ScheduledTasks.TaskManager: Attempting to cancel Scheduled Task Generate BIF Files
jellyfin    | [14:01:39] [ERR] [82] Nick.Plugin.Jellyscrub.ScheduledTasks.BIFGenerationTask: Error creating trickplay files for tape 7  03-04-1999---25-02: System.ComponentModel.Win32Exception (13): Permission denied
jellyfin    |    at System.Diagnostics.Process.set_PriorityClassCore(ProcessPriorityClass value)
jellyfin    |    at System.Diagnostics.Process.set_PriorityClass(ProcessPriorityClass value)
jellyfin    |    at Nick.Plugin.Jellyscrub.Drawing.OldMediaEncoder.StartProcess(ProcessWrapper process)
jellyfin    |    at Nick.Plugin.Jellyscrub.Drawing.OldMediaEncoder.ExtractVideoImagesOnInterval(String inputFile, String container, MediaStream videoStream, MediaSourceInfo mediaSource, Nullable`1 threedFormat, TimeSpan interval, String targetDirectory, String filenamePrefix, Int32 maxWidth, CancellationToken cancellationToken)
jellyfin    |    at Nick.Plugin.Jellyscrub.Drawing.VideoProcessor.CreateBif(String path, Int32 width, Int32 interval, BaseItem item, MediaSourceInfo mediaSource, CancellationToken cancellationToken)
jellyfin    |    at Nick.Plugin.Jellyscrub.Drawing.VideoProcessor.Run(BaseItem item, MediaSourceInfo mediaSource, Int32 width, Int32 interval, CancellationToken cancellationToken)
jellyfin    |    at Nick.Plugin.Jellyscrub.Drawing.VideoProcessor.Run(BaseItem item, CancellationToken cancellationToken)
jellyfin    |    at Nick.Plugin.Jellyscrub.ScheduledTasks.BIFGenerationTask.ExecuteAsync(IProgress`1 progress, CancellationToken cancellationToken)

I'm running the official Docker image on Debian Server (Buster) with user: 1000:1000.

Also if you have on demand generation on you can click on the dolby vision video and it should start generation. Although I guess you can't test it if 1.0.0.3 isnt working at all

The color mapping isn't working with 1.0.0.3 as well.

[nicknsy]

So did the trickplay files still generate on 1.0.0.3 even will the error?

[DigitalWriting-Dev]

So did the trickplay files still generate on 1.0.0.3 even will the error?

Maybe I didn't explain myself correctly, trickplay files only get created when Process Priority in the plugin settings is set to Normal or lower. Above Normal and High give the permission error.

[nicknsy]

Ah okay. Could you post the output of the "ffprobe" command on the HDR file that isn't being tonemapped correctly? I'm not sure what the problem would be as the hdr tonemapping filter was taken directly from the chapter image generation in jellyfin.

[DigitalWriting-Dev]

Ah okay. Could you post the output of the "ffprobe" command on the HDR file that isn't being tonemapped correctly? I'm not sure what the problem would be as the hdr tonemapping filter was taken directly from the chapter image generation in jellyfin.

I'm pretty sure Jellyfin generates chapter images without color mapping, the regular chapter thumbnails have the same issue.

ffprobe version 4.1.9-0+deb10u1 Copyright (c) 2007-2022 the FFmpeg developers
  built with gcc 8 (Debian 8.3.0-6)
  configuration: --prefix=/usr --extra-version=0+deb10u1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Input #0, matroska,webm, from '/vault/Media/Movies/Minions The Rise of Gru (2022)/Minions The Rise of Gru (2022) - DV.mkv':
  Metadata:
    encoder         : libebml v1.4.2 + libmatroska v1.6.4
    creation_time   : 2022-08-02T04:14:06.000000Z
  Duration: 01:27:55.74, start: 0.000000, bitrate: 25154 kb/s
    Chapter #0:0: start 0.000000, end 141.558000
    Metadata:
      title           : Chapter 01
    Chapter #0:1: start 141.558000, end 458.083000
    Metadata:
      title           : Chapter 02
    Chapter #0:2: start 458.083000, end 767.684000
    Metadata:
      title           : Chapter 03
    Chapter #0:3: start 767.684000, end 1070.528000
    Metadata:
      title           : Chapter 04
    Chapter #0:4: start 1070.528000, end 1246.329000
    Metadata:
      title           : Chapter 05
    Chapter #0:5: start 1246.329000, end 1580.538000
    Metadata:
      title           : Chapter 06
    Chapter #0:6: start 1580.538000, end 1741.658000
    Metadata:
      title           : Chapter 07
    Chapter #0:7: start 1741.658000, end 2013.430000
    Metadata:
      title           : Chapter 08
    Chapter #0:8: start 2013.430000, end 2329.454000
    Metadata:
      title           : Chapter 09
    Chapter #0:9: start 2329.454000, end 2603.770000
    Metadata:
      title           : Chapter 10
    Chapter #0:10: start 2603.770000, end 2939.773000
    Metadata:
      title           : Chapter 11
    Chapter #0:11: start 2939.773000, end 3027.068000
    Metadata:
      title           : Chapter 12
    Chapter #0:12: start 3027.068000, end 3339.631000
    Metadata:
      title           : Chapter 13
    Chapter #0:13: start 3339.631000, end 3600.975000
    Metadata:
      title           : Chapter 14
    Chapter #0:14: start 3600.975000, end 3933.058000
    Metadata:
      title           : Chapter 15
    Chapter #0:15: start 3933.058000, end 4060.977000
    Metadata:
      title           : Chapter 16
    Chapter #0:16: start 4060.977000, end 4258.133000
    Metadata:
      title           : Chapter 17
    Chapter #0:17: start 4258.133000, end 4586.169000
    Metadata:
      title           : Chapter 18
    Chapter #0:18: start 4586.169000, end 4771.104000
    Metadata:
      title           : Chapter 19
    Chapter #0:19: start 4771.104000, end 5275.741000
    Metadata:
      title           : Chapter 20
    Stream #0:0(eng): Video: hevc (Main 10), yuv420p10le(tv), 3832x1596 [SAR 1:1 DAR 958:399], 23.98 fps, 23.98 tbr, 1k tbn, 23.98 tbc (default)
    Metadata:
      BPS             : 24387869
      DURATION        : 01:27:55.741000000
      NUMBER_OF_FRAMES: 125931
      NUMBER_OF_BYTES : 16083010641
      _STATISTICS_WRITING_APP: mkvmerge v61.0.0 ('So') 64-bit
      _STATISTICS_WRITING_DATE_UTC: 2022-08-02 04:14:06
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
    Stream #0:1(eng): Audio: eac3, 48000 Hz, 6 channels, fltp (default)
    Metadata:
      BPS             : 768000
      DURATION        : 01:27:31.168000000
      NUMBER_OF_FRAMES: 164099
      NUMBER_OF_BYTES : 504112128
      _STATISTICS_WRITING_APP: mkvmerge v61.0.0 ('So') 64-bit
      _STATISTICS_WRITING_DATE_UTC: 2022-08-02 04:14:06
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
    Stream #0:2(spa): Subtitle: subrip
    Metadata:
      title           : Latin American
      BPS             : 34
      DURATION        : 01:26:42.578000000
      NUMBER_OF_FRAMES: 949
      NUMBER_OF_BYTES : 22487
      _STATISTICS_WRITING_APP: mkvmerge v61.0.0 ('So') 64-bit
      _STATISTICS_WRITING_DATE_UTC: 2022-08-02 04:14:06
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
    Stream #0:3(fre): Subtitle: subrip
    Metadata:
      title           : Canadian
      BPS             : 37
      DURATION        : 01:26:15.467000000
      NUMBER_OF_FRAMES: 936
      NUMBER_OF_BYTES : 24242
      _STATISTICS_WRITING_APP: mkvmerge v61.0.0 ('So') 64-bit
      _STATISTICS_WRITING_DATE_UTC: 2022-08-02 04:14:06
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES

Also, with 1.0.0.4 the same error happens (again, with High priority selected):

jellyfin    | [14:41:13] [INF] [15] Nick.Plugin.Jellyscrub.Drawing.VideoProcessor: Creating trickplay files at 320 width, for /homevideos/קלטות/tape 8  25-02-2000---15-10-2000.mkv [ID: df4f3572-b23b-f65b-be22-7b42a51b2175]
jellyfin    | [14:41:13] [INF] [15] Nick.Plugin.Jellyscrub.Drawing.OldMediaEncoder: /usr/lib/jellyfin-ffmpeg/ffmpeg -f matroska,webm -i file:"/homevideos/קלטות/tape 8  25-02-2000---15-10-2000.mkv" -threads 0 -v quiet -filter:v fps=1/10,scale=min(iw\,320):trunc(ow/dar/2)*2 -f image2 "/config/cache/temp/420a27cc43c4490a8171a46c61362873/img_%08d.jpg"
jellyfin    | [14:41:13] [ERR] [15] Nick.Plugin.Jellyscrub.Drawing.OldMediaEncoder: Unable to set process priority: Permission denied

This time though, it's not trying to continue to the next file.

[nicknsy]

Yeah that error message is intentional. What matters is that it’s not skipping generation.

As for HDR, it makes sense now why it wouldn’t work if it didn’t work with Jellyfin’s chapter images. I guess I have to try and find a separate way to do it.

[DigitalWriting-Dev]

Yeah that error message is intentional. What matters is that it’s not skipping generation.

As for HDR, it makes sense now why it wouldn’t work if it didn’t work with Jellyfin’s chapter images. I guess I have to try and find a separate way to do it.

Fair enough. Jellyfin 10.8.0 now has the option to color map Dolby Vision using their custom ffmpeg version. If you need help with some Dolby Vision files to test this, there are a bunch of them here.

[DigitalWriting-Dev]

I did a few tests with manual "color mapping" and it looks like just setting the hue to +100 does the trick. It's absolutely not perfect, and some scenes look better than others, but I think it's good enough for a thumbnail, and it shouldn't take too much processing power. This is the ffmpeg filter I've added: -vf 'hue=h=100:s=1.3'.

[nicknsy]

Just adding this as a note to myself: this is an issue with DV profile 5 (maybe 8 sometimes?) because there is not a fallback layer when playing HDR content on SDR screens. This was fixed in jellyfin for video playback in this commit but image extraction in jellyfin is still broken for DV. Still, the same tonemap should work fine for the bif extraction.