Preserve DV in handbrake transcodes?

[bjaurelio]

I have an UnRaid server that I use to rip movies with MakeMKV. The file sizes are very large for 4k movies, but with handbrake, they can be reduced substantially with no loss in quality. The problem with using it on Dolby vision titles, is that info gets lost. I ran across dovi_tool and was looking for a docker container that uses it.

With Dobby, could I put both files in the same folder and get an output that uses the smaller h.265 handbrake video file and injects the DV rpu from the original?

[Zggis]

Hi bjaurelio,

I believe you can, but I have not tested that exact use case. Dobby does use dovi_tool for the injection.

I would recommend giving it a shot, and let me know how it goes. If it doesn't work upload your dobby.log file here and I will try to reproduce on my end. The important thing is Dolby vision must be completely absent from your post-handbrake file. If Dobby finds signs of DV data on both files it will get confused and might try to merge in the wrong direction. Dobby uses MediaInfo to determine if a file has DV on it, so you can check with that to confirm.

[bjaurelio]

Thanks for the quick reply. I'm going to try it out with Dune once handbrake finishes it and let you know my findings. So far I have just been focused on standard blu-rays and hdr10 only titles. I am also going to see if I can use the intel hw acceleration by using "-hwaccel qsv" and adding the device to the container.

[bjaurelio]

Here's my log. It ignored the original file because it has both DV and HDR. As a result, it was unable to find an RPU. I do have an RPU because I found a tool that I can run on my windows machine to extract it as a .bin file. Even placing that in the folder did not work. dobby.log

[Zggis]

Got it. I added that check because sometimes I would not realize a file already had DV + HDR, its proved helpful to me so I am going to leave it, but I will see if I can add a new variable to disable it, this would allow your use case.

I already have a copy of Dune BluRay and will install handbrake to see if I can get this to work. In meantime, if you can upload the Handbrake settings you are using for your Dune conversion that might help. (I have never used Handbrake before, but will use your above comments to try and match what you are doing)

FYI: Placing the RPU .bin file in Dobby's directory won't do anything, Dobby will only look at MP4 and MKV files. It will extract its own RPU .bin file during conversion. So if you had a Dune MP4 or MKV that was DV only you could pair that with the post handbrake file and it should work, but I doubt you have that if you ripped the movie with MakeMKV. This is the use case Dobby was designed for, its common to find separate files for HDR vs DV on the streaming services.

[bjaurelio]

I have a Dune mkv that I ripped using MakeMKV. It contains both HDR and DV from the disk. In handbrake, I'm using mkv format, h265 10 bit, cq 10, and passing through the audio. With handbrake, you lose the DV info. I'm trying to inject the DV back into the significantly smaller file. It works great for standard blu-rays and non-DV 4K titles because the size reduction is significant with no noticeable drop in quality. I spent a long time comparing the two files for How to Train Your Dragon trying to pick apart differences in shadows and highlights before deciding to go forward because it cuts the size down to less than half.

[Zggis]

Ok I was able to get this to work on my side. I have added a new configuration variable ACCEPTBLRPUINPUT if you set this to true it will allow your original DV+HDR file to be used as the DV file and the converted HDR file as the target.

I pushed v2.0.1 to Dockerhub latest, you will need to force update your app, Unraid only checks for updates every few hours or so. The version is printed in the log file for confirmation. The README has also been updated, you can see all the configuration options there.

I also had to set AAVALIDATE to false since my conversion had enough discrepancies where the frame heights were not lining up, and the validation was failing. I suspect you will have to do the same.

I also highly recommend you name your files with the title in them, ex. 'Dune.original.mkv' & 'Dune.converted.mkv', Dobby's matching algorithm uses frame count and filename, since your files are converted the frame count may not be an option so it would help the matching algorithm if it can match the title in the filename.

In my Dune test my original file was 1.77GB (I cut it down to a 3 min clip so handbrake wouldn't murder my CPU) and my post handbrake conversion was 66MB. After the DV inject Dobby gave a file that was 67MB and I was able to verify the result had DV data with MediaInfo. My conversion configuration might be a bit different but I think it should still work.

[Zggis]

[bjaurelio]

It initially failed, so I had to change the validation to false. It then stated that it failed because it did not detect DV, but when I played it through Jellyfin, the new file came up with DV on my LG C9. Thank you so much. This is going to save me a ton of space. Dune was cut down by 64% on the file size. I had to nitpick at frames to try and discern any difference. With motion, there's no way I would detect a difference between the two side by side.

[Zggis]

The active area validation is expected to fail in this case. Glad the result worked for you.

I think I know why the result validation failed, the result needs to have a HDR format of 'SMPTE ST 2086' or 'SMPTE ST 2094' for that to pass. I suspect your result might be Dolby Vision + 'SMPTE ST 2084' which should still be HDR compatible, so I will add it to the result validation.

If you can confirm the HDR SMPTE format of your result that would help me confirm this. Thanks!

[bjaurelio]

How do I confirm this? I tried looking at it using media info, but that does not appear to show an SMPTE ST for the file.

[Zggis]

If you open the file in MediaInfo and go to View > Text and look for the HDR Format line it should be there.

You can also go to File > Export and export as Text and post the output here.

[bjaurelio]

Here's what I have. There's no SMPTE ST listed.

[Zggis]

Thanks, I updated the result validation in v2.0.2, it doesn't look for SMPTE anymore, just HDR10 compatibility and DV. Hopefully that fixes the validation step for you.

[PinoPostino155]

Hi i'm use zggis from docker desktop and i want merge flie with dv+hdr10+ to file with hdr. But the terminal tell me this Dolby Vision and HDR were already detected on /data/media/Avatar.2009.2160p.UHD.BluRay.DV.HDR10.DDP.5.1.x265-BiTOR.mkv, this file will be ignored. You can allow this file by setting ACCEPTBLRPUINPUT to true, and I don't know what to do can someone help me, thanks and good evening.

[Zggis]

Hi,

The Dobby application will take a DV file and merge it into an HDR file to create a file with both DV and HDR. In your case, it looks like your Avatar file already has DV+HDR so Dobby skips it.

As described above, when a DV+HDR file is converted using handbrake, the DV layer is lost. In these cases Dobby can be used to restore the DV layer on the handbrake result by merging the original DV+HDR file into the handbrake result, but only if you set ACCEPTBLRPUINPUT to true. If this is your use case than you can try setting ACCEPTBLRPUINPUT to true and your HDR file should get the DV layer from Avatar.2009.2160p.UHD.BluRay.DV.HDR10.DDP.5.1.x265-BiTOR.mkv

If you are using Docker desktop you can try this run command: docker run -v MEDIA_DIR:/data/media zggis/dobby:latest -e ACCEPTBLRPUINPUT='true'

[PinoPostino155]

Hi,

The Dobby application will take a DV file and merge it into an HDR file to create a file with both DV and HDR. In your case, it looks like your Avatar file already has DV+HDR so Dobby skips it.

As described above, when a DV+HDR file is converted using handbrake, the DV layer is lost. In these cases Dobby can be used to restore the DV layer on the handbrake result by merging the original DV+HDR file into the handbrake result, but only if you set ACCEPTBLRPUINPUT to true. If this is your use case than you can try setting ACCEPTBLRPUINPUT to true and your HDR file should get the DV layer from Avatar.2009.2160p.UHD.BluRay.DV.HDR10.DDP.5.1.x265-BiTOR.mkv

If you are using Docker desktop you can try this run command: docker run -v MEDIA_DIR:/data/media zggis/dobby:latest -e ACCEPTBLRPUINPUT='true'

Hi thank, but i want merge hdr 10+ and dolbyvision into hdr file. My file mkv was extracted by makemkv and i don't use the handrbrake software

[PinoPostino155]

And the terminal say me this: Unable to find a RPU match for /data/media/dobbyTemp/Avatar.2009.2160p.UHD.REMUX.HDR10.HEVC.Atmos.7.1-ARENA.hevc, check the logs above. This may be caused by a missing file, or a file that failed validation and was skipped as a result. and Unable to find a BLRPU match for /data/media/Avatar.2009.2160p.UHD.REMUX.HDR10.HEVC.Atmos.7.1-ARENA.mkv, check the logs above. This may be caused by a missing file, or a file that failed validation and was skipped as a result.

[Zggis]

Dobby does not officially support HDR10+, only Doby Vision.

The error you posted indicates the application can't match your two files. In your media directory you should have Avatar.2009.2160p.UHD.REMUX.HDR10.HEVC.Atmos.7.1-ARENA AND the hdr file you are trying to merge it into.

The matching algorithm looks at the file names, so both files should start with 'Avatar.2009...' to ensure a match is found.

If the issue persist please upload your full dobby.log file