CrendKing / avisynth_filter

DirectShow filters that put AviSynth and VapourSynth into video playing
MIT License
108 stars 8 forks source link

Dolby Vision support #87

Open chainikdn opened 1 year ago

chainikdn commented 1 year ago

Latest MPC Video Renderer supports Dolby Vision, so it's a good idea to pass DoVi data through AVSF. I don't want to make any PRs cause you'll rewrite everything from scratch anyway :D

There're number of simple-to-solve issues here:

  1. Update side_data.h, deal with IID_MediaSideDataDOVIMetadata and IID_MediaSideDataDOVIRPU.

  2. DolbyVision meta data is a color curves defined for each frame. These curves can occasionally change multiple times in one scene which creates a problem for frame interpolation, cause interpolation wants to merge two (or more) frames, and these frames are different in colors. So my idea is AVSF must propagate DoVi data to AVS side so the interpolation filter could do something about this case.

2.1. Put DoVi data into source frame's property (the same for Avisynth, obviously 3.6+ only):

            hdrSideData->ReadFrom(inputSampleSideData);
            .....
            const BYTE* doviData;
            size_t doviSz = 0;
            hdrSideData->RetrieveSideData(IID_MediaSideDataDOVIMetadata, &doviData, &doviSz);
            if (doviSz > 0)
                AVSF_VPS_API->mapSetData(frameProps, "_DoVi", (const char*)doviData, (int)doviSz, dtBinary, maReplace);

2.2. It's very important that we take side data from the exact frame number at the output, so we need to fix Avisynth Filter, make it work the same way as Vapousynth by setting "AVSF_SourceFrameNb" and grab correct side data like this:

                     sourceFrameNb = static_cast<int>(AVSF_AVS_API->propGetInt(frameProps, FRAME_PROP_NAME_SOURCE_FRAME_NB, 0, &propGetError));
                     ................
                if (const ATL::CComQIPtr<IMediaSideData> sideData(outSample); sideData != nullptr) {
                    if (sourceFrameNb >= 0) { //must be always set with AVS 3.6+
                        const auto iter = _sourceFrames.find(sourceFrameNb);
                        ASSERT(iter != _sourceFrames.end());

                        iter->second.hdrSideData->WriteTo(sideData);
                    }
                    else
                        processSourceFrameIters[0]->second.hdrSideData->WriteTo(sideData);
                }
chainikdn commented 1 year ago

(*) "curves can occasionally change multiple times in one scene" - what I mean is: w/o applying DoVi curves, raw colors in 1st frame are more, for example, pink-ish then in 2nd one, but after applying curves in the renderer they will look the same.

(**) the idea is interpolation filter won't merge two frames when DoVi curves change. Instead it will interpolate using only one frame.

CrendKing commented 1 year ago

I'm sorry if I gave you the impression that I repel PRs. Based on what you have put down here, you clearly have expertise on this topic, so a PR is welcomed.