MKV file: Image resolution used instead of video resolution

[StephKoenig]

I've noticed in mkv files which can contain both video and images (for example videos with a cover image) VDF returns the image resolution instead to video resolution. Of course this is not necessarily wrong since VDF can also scan for images but I would suggest if both video and images are found, that the video resolution is used UI presentation.

I would also be handy to have sort by first name (meaning the name of the first file in group). I appreciate for some this might not be useful but for example where you have a folder full of clips where VFD cannot be the sole arbiter say with a 50-80% match setting going through them in order is faster that arbitrarily jumping around the folder.

A 'close' match is much more useful than the standard binary matches especailly when dealing with multipel resolutions and different pre and post-rolls...

Lastly would be handy to have the 'system' menu available on a context click.

~Steph

[strazto]

This is really a few separate issues, though I agree with everything you've raised.

I've noticed in mkv files which can contain both video and images (for example videos with a cover image) VDF returns the image resolution instead to video resolution. Of course this is not necessarily wrong

Sounds problematic - I'd argue that this is probably almost always wrong, since no-one really uses the MKV container to store images, and there are plenty of other duplicate finders, that are more mature than this one, that support image comparisons.

I would also be handy to have sort by first name

Sorting by file name would be useful, I agree - in general there needs to be more ways to sort

A 'close' match is much more useful than the standard binary matches especailly when dealing with multipel resolutions and different pre and post-rolls...

Not sure what this one pertains to

Lastly would be handy to have the 'system' menu available on a context click.

What do you mean by that? As in, the right click menu that appears in your file explorer?

[StephKoenig]

Thanks for the reply.. great tool

On the MKV mixed mode I agree VDF should only focus on the video tracks and then probably only on the first one (or default one) ... images are very rare and typically only on mkvs if they're used as covers/box art... MetaX and Icaros both provide for cover support creation and display which works really well if interested.

Re: Close match, I can't remember now, maybe I was just saying for my usecase that its more useful than exact - for example a folder full of edits with include/exclude pre and post roll and where you have multiple resolutions... exact match almost never happens and anyway there's plenty of binary comparison tools that can do that already.

Re: System menu.. thats right I meant importing or making available the right click/context click content you might get with your OS on that file type... if its tricky to add withing to a sub menu then maybe a alt-right-clicking can display it solo...

[0x90d]

VDF uses ffmpeg to get the thumbnails. And this works in 99% of all cases. Checking all video files for a video stream first for such a ninche scenario doesn't sound like a good idea, at least not by default. Perhaps as an opt-in but this would definitely need a pull request as I have no plan to add it myself.

[StephKoenig]

Sure I get that but I think there a slight misunderstanding... the only issue is that VDF GUI doesn't report the dimensions of the video track it has used for comparison instead it reports the dimenions of the first object inside the MKV, this is sometimes an image (in a very small number of cases). Not suggesting the VFD support cover art at all...

As you say super niche for some to store a cover image inside a MKV file. Results are great esp in OSs that support it... https://imgur.com/a/VHjM4Sf

Love VDF.

[Maltragor]

As I understand it, thumbnail generation (+duplicate detection) and getting the media information runs completely independently.

For the thumbnails, ffmpeg is called with a given scale / resolution, and even then selects a video (and audio) stream deemed suitable and generates an image.

For the media information, on the other hand, ffprobe is called and the relevant data of all found streams is cached. Later (constructor of DuplicateItem), all streams are then gone through and a (arbitrary) audio and video stream is selected whose data is displayed. (The chosen implementation ensures that the last stream of the respective type is selected. But of course this is not necessarily the one ffmpeg has decided for).

According to which criteria ffmpeg decides for a stream, I don't know. And off the top of my head I don't know how to tell ffprobe to select something according to the same criteria. What in principle would still be possible is:

/usr/bin/ffmpeg -i "VideoFile.mp4" 2>&1 | grep "Stream" This outputs something like

    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 3975 kb/s, 60 fps, 60 tbr, 16k tbn, 120 tbc (default)
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 189 kb/s (default)

and possibly only the selected streams are included. (I don't have a video with multiple streams of the same type right now to check).

[StephKoenig]

Firstly totally appreciate this is an edge case so thanks for even looking at this...

If it helps here's a tiny example: https://1drv.ms/u/s!AnTJ6eIXZViJvwRvzLkKCWjHka6c?e=ph84Nu The 25MB mkv is x265 1080p with a 700x986 cover art embedded as cover.jpg. You can manually do this via mkvtoolnix on the attachement tab or there's a few tools to automate the file tagging and a windows shell extension to display the cover art like mp4, mp3 and comics called Icaros.

Anyway to your point I suppose the thumbnailer is reading the coverart and then in the gui the file is reported to be 700x986 .. I guess just dupe up the mkv (it's only 27MBs) a few times to see it in action.

Here's a screen shot of what it does with those two files https://imgur.com/78T64bj You can probably also see it's decided the encoding is mjpeg

[Maltragor]

Calling ffprobe to get the media infos, what VDF will see:

/usr/bin/ffprobe -hide_banner -loglevel error -print_format json -sexagesimal -show_format -show_streams "./Wonders of the Universe s01e01 Destiny.mkv"

(I have added some marks (*, #, ###) and some non-relevant sections abbreviated)

"streams": [
        {
            "index": 0,
            "codec_name": "hevc",
            "codec_long_name": "H.265 / HEVC (High Efficiency Video Coding)",
           "profile": "Main",
        *   "codec_type": "video",
        #   "codec_time_base": "1/25",
            "codec_tag_string": "[0][0][0][0]",
            "codec_tag": "0x0000",
        *   "width": 1920, 
        *   "height": 1080, 
            "coded_width": 1920,
            "coded_height": 1080,
            "has_b_frames": 2,
            "sample_aspect_ratio": "1:1",
            "display_aspect_ratio": "16:9",
            "pix_fmt": "yuv420p",
            "level": 120,
            "color_range": "tv",
            "color_space": "bt709",
            "color_transfer": "bt709",
            "color_primaries": "bt709",
            "refs": 1,
            "r_frame_rate": "25/1",
        #   "avg_frame_rate": "25/1", 
            "time_base": "1/1000",
            "start_pts": 80,
            "start_time": "0:00:00.080000",
            "disposition": {
      ###       "default": 1,
                "dub": 0,
                "original": 0,
                "comment": 0,
                "lyrics": 0,
                "karaoke": 0,
                "forced": 0,
                "hearing_impaired": 0,
                "visual_impaired": 0,
                "clean_effects": 0,
        #       "attached_pic": 0,
                "timed_thumbnails": 0
            },
            "tags": {...}
        },
        {
            "index": 1,
            ...
            "codec_type": "audio",
            ...
        },
        {
            "index": 2,
            "codec_name": "mjpeg",
            "codec_long_name": "Motion JPEG",
            "profile": "Progressive",
        *   "codec_type": "video", 
        #   "codec_time_base": "0/1",
            "codec_tag_string": "[0][0][0][0]",
            "codec_tag": "0x0000",
        *   "width": 700,
        *   "height": 986,
            "coded_width": 700,
            "coded_height": 986,
            "has_b_frames": 0,
            "sample_aspect_ratio": "1:1",
            "display_aspect_ratio": "350:493",
            "pix_fmt": "yuvj420p",
            "level": -99,
            "color_range": "pc",
            "color_space": "bt470bg",
            "chroma_location": "center",
            "refs": 1,
            "r_frame_rate": "90000/1",
        #   "avg_frame_rate": "0/0", 
            "time_base": "1/90000",
            "start_pts": 7200,
            "start_time": "0:00:00.080000",
            "duration_ts": 22944240,
            "duration": "0:04:14.936000",
            "bits_per_raw_sample": "8",
            "disposition": {
      ###       "default": 0,
                "dub": 0,
                "original": 0,
                "comment": 0,
                "lyrics": 0,
                "karaoke": 0,
                "forced": 0,
                "hearing_impaired": 0,
                "visual_impaired": 0,
                "clean_effects": 0,
         #      "attached_pic": 1,
                "timed_thumbnails": 0
            },
            "tags": {...}
        }
    ],
    "format": {
        "filename": "./Wonders of the Universe s01e01 Destiny.mkv",
        "nb_streams": 3,
        "nb_programs": 0,
        "format_name": "matroska,webm",
        "format_long_name": "Matroska / WebM",
        "start_time": "0:00:00.080000",
        "duration": "0:04:14.936000",
        "size": "27910567",
        "bit_rate": "875845",
        "probe_score": 100,
        "tags": {...}
    }
}

So there are two streams of codec_type `"video". VDF always uses the last record of a certain type as I described recently, so here it decides for "index": 2 and thus the cover image which is embedded as mjpeg encoded.

Try out what ffmpeg outputs...

/usr/bin/ffmpeg -i "./Wonders of the Universe s01e01 Destiny.mkv" 2>&1 | grep "Stream"
    Stream #0:0: Video: hevc (Main), yuv420p(tv, bt709), 1920x1080, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 1k tbn, 25 tbc (default)
    Stream #0:1: Audio: aac (LC), 48000 Hz, stereo, fltp (default)
    Stream #0:2: Video: mjpeg (Progressive), yuvj420p(pc, bt470bg/unknown/unknown), 700x986 [SAR 72:72 DAR 350:493], 90k tbr, 90k tbn, 90k tbc (attached pic)

OK, so both video streams are also listed, but one has a (default) marker. I assume this corresponds to disposition.default=1.

@0x90d If my assumption with disposition.default is correct, one would have to either include disposition.default in the FFProbeJsonReader (or already discard all non-default streams here). And in the DuplicateItem in the loop prevent that the info variables are overwritten further, after the data of the default stream were assigned.

[StephKoenig]

@Maltragor nice find... I looked through several mkvs with and without the jpg embedded... it seems the main video is always default unless overridden with mkvtools - i"m guessing because it was the first track muxed. I also built some new mkvs and the first video tracks is always marked default.

Since you never create an mkv with just a cover/mjpeg 'video track' before adding the real video tracks I think you've found a good suggestion which is to select the dispostion.default video track (and discard the rest) if present otherwise use original selection method.