Lossless crop and video bitstream or container format metadata

[mifi]

MKV

https://github.com/mifi/lossless-cut/issues/504#issuecomment-773986073

Actually, this is possible with a Matroska container. I've done it multiple times in the past with MKVToolNix GUI, and it plays perfectly fine in MPC-BE. For not supporting players like VLC, bug reports should be filed.

The only caveat is that the video data stays the same and the file size is not reduced. But for playing purposes, it's very useful.

In MKVToolNix GUI, import a video file, click the video track, and hover over the text box of "Video properties > Cropping" for help. There's no visual cropping tool, though. That's why Lossless Cut should have it.

I wish this would be implemented. You'd just have to communicate that the feature is limited to MKV output, and that it's just the cropping information that is saved, so the file size isn't reduced and the player is responsible for the actual cropping.

If FFMPEG is the problem (can't it set metadata in the output file?), maybe it must be adapted or a multi-tool solution with the command line equivalent of the MKVToolNix GUI header editor might be the way to go.

Need to figure out how to do this with ffmpeg.

EDIT: added h264:

H264

-bsf:v h264_metadata

https://github.com/mifi/lossless-cut/issues/38#issuecomment-1864479088

[h-h-h-h]

Note that, in MKVToolNix GUI, additionally to "Cropping", "Display width/height" must be set to the new video size for distortion-free cropping (should also be possible via "Set aspect ratio").

(This brings us to the idea that Matroska medadata would also make possible the following: aspect ratio correction [spacial stretching/compressing]; adjusting audio delay; temporal stretching/compressing by setting stretch factors and/or FPS per track [even for synced combination of, say, 24-fps video track with English audio track of file 1 and sped-up German audio track of 25-fps video file 2].)

[shodanx2]

Hello, I would like to signal interest in this feature. I was searching for a "lossless crop" app as efficient as losslesscut but that doesn't appear to exist yet ! So, an arbitrary crop is not possible without a transcode because of the partial blocks where you're cropping. For that reason, I think a limited crop that can only crop whole quantization blocks, could that work ?

You'd lose some accuracy on the crop dimensions depending on the size of the blocks, but I think that would still be very usable way to crop video and the benefits of not having to transcode are massive

[Craftyawesome]

For an alternative, some codecs like H264 and HEVC also have crop metadata. E.g. for H264 ffmpeg -i inputfile.mp4 -bsf:v h264_metadata=crop_left=128:crop_right=128:crop_top:0:crop_bottom=0 -codec copy outputfile.mp4 It actually seems to work in a lot of players, although VLC gets it wrong

[shodanx2]

Wow that would be great ! I just needed to re-frame a scene and didn't want to recompress, losing quality, this would be perfect, to draw a crop rectangle on the screen and move it around at various key frames !

[mifi]

nice find! i wonder if it works on the most popular video players and tools like

• chrome's built-in html5 video player
• safari's & ios safari's built in html5 video player
• firefox html5 video
• iphone photos app
• android built in video player
• android photos
• apple quicktime
• apple quicklook
• windows media player
• ffmpeg (e.g. read a losslessly cropped video and re-encode it, observing that the encoded video is cropped correctly)

[shodanx2]

Hello,

Please note where was a typo in @Craftyawesome 's command

ffmpeg -i inputfile.mp4 -bsf:v h264_metadata=crop_left=128:crop_right=128:crop_top:0:crop_bottom=0 -codec copy outputfile.mp4

should be

ffmpeg -i inputfile.mp4 -bsf:v h264_metadata=crop_left=128:crop_right=128:crop_top=0:crop_bottom=0 -codec copy outputfile.mp4

FFmpeg output

ffmpeg -hide_banner -i testvideo.mp4 -bsf:v h264_metadata=crop_left=128:crop_right=128:crop_top=0:crop_bottom=0 -codec copy testvideo.metacropped.mp4
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'testvideo.mp4':
  Metadata:
    major_brand     : dash
    minor_version   : 0
    compatible_brands: iso6avc1mp41
    creation_time   : 2023-02-14T13:25:33.000000Z
  Duration: 00:01:09.74, start: 0.000000, bitrate: 1985 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 7 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)
    Metadata:
      creation_time   : 2023-02-14T13:25:33.000000Z
      handler_name    : ISO Media file produced by Google Inc.
      vendor_id       : [0][0][0][0]
Output #0, mp4, to 'testvideo.metacropped.mp4':
  Metadata:
    major_brand     : dash
    minor_version   : 0
    compatible_brands: iso6avc1mp41
    encoder         : Lavf60.11.100
  Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 7 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)
    Metadata:
      creation_time   : 2023-02-14T13:25:33.000000Z
      handler_name    : ISO Media file produced by Google Inc.
      vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
[out#0/mp4 @ 0000025d8cfe5200] video:16874kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.110405%
frame= 2090 fps=0.0 q=-1.0 Lsize=   16893kB time=00:01:09.66 bitrate=1986.3kbits/s speed=1.48e+03x

mediainfo

vlc

Hmm, I might need a more concrete test video, but it's doing something

[Craftyawesome]

Please note where was a typo in @Craftyawesome 's command

My bad, copied it from one of the related issues in that vlc issue link without checking myself.

Hmm, I might need a more concrete test video, but it's doing something

As mentioned in the issue link vlc seems to interpret left and top as an additional right and bottom. Try almost any other player, it seems to be quite compatible barring vlc.

[shodanx2]

Ok,

I have found a more suitable test file

https://github.com/mifi/lossless-cut/assets/10621885/a5ca8d7e-3452-4dff-998f-a3df41e6a855

I have run the following commands to crop it different ways for testing

ffmpeg -hide_banner -i testvideo.mp4 -bsf:v h264_metadata=crop_left=500:crop_right=0:crop_top=0:crop_bottom=0 -codec copy testvideo.metacropped.cropleft500.mp4
ffmpeg -hide_banner -i testvideo.mp4 -bsf:v h264_metadata=crop_left=0:crop_right=500:crop_top=0:crop_bottom=0 -codec copy testvideo.metacropped.cropright500.mp4
ffmpeg -hide_banner -i testvideo.mp4 -bsf:v h264_metadata=crop_left=0:crop_right=0:crop_top=500:crop_bottom=0 -codec copy testvideo.metacropped.croptop500.mp4
ffmpeg -hide_banner -i testvideo.mp4 -bsf:v h264_metadata=crop_left=0:crop_right=0:crop_top=0:crop_bottom=500 -codec copy testvideo.metacropped.cropbottom500.mp4
ffmpeg -hide_banner -i testvideo.mp4 -bsf:v h264_metadata=crop_left=250:crop_right=250:crop_top=0:crop_bottom=0 -codec copy testvideo.metacropped.cropmiddlehorizon.mp4
ffmpeg -hide_banner -i testvideo.mp4 -bsf:v h264_metadata=crop_left=0:crop_right=0:crop_top=250:crop_bottom=250 -codec copy testvideo.metacropped.cropmiddlevertical.mp4
ffmpeg -hide_banner -i testvideo.mp4 -bsf:v h264_metadata=crop_left=250:crop_right=250:crop_top=250:crop_bottom=250 -codec copy testvideo.metacropped.cropcenter.mp4

Here is the console output of that ffmpeg-metacrop.log

And here are the test files

https://github.com/mifi/lossless-cut/assets/10621885/6f7e6394-2826-48b8-b6d4-e93b84cd4143

https://github.com/mifi/lossless-cut/assets/10621885/3cb15933-c415-4fb1-b7ca-0f00cc450ddf

https://github.com/mifi/lossless-cut/assets/10621885/febed6a7-d734-4339-b289-ddeca66ea266

https://github.com/mifi/lossless-cut/assets/10621885/bd8c4556-3249-4925-9950-33e05b4b2e03

https://github.com/mifi/lossless-cut/assets/10621885/14e331ca-6872-4426-a7af-74d4e0c729eb

https://github.com/mifi/lossless-cut/assets/10621885/ce0b68d0-1008-41f7-938f-8f727b1052e1

https://github.com/mifi/lossless-cut/assets/10621885/48a96627-b08f-4e7d-8862-0d02006f598a

Here is mediainfo about these files.

[shodanx2]

Here is how VLC handles those files.

[shodanx2]

Here is how MPC-HC abandonned version MPC-HC.1.7.13.x64.exe handles it

[shodanx2]

Here is how MPC-HC.1.9.24.x64.exe handles it

[shodanx2]

Here is how MPC-HC.2.0.0.x64.exe handles it

[shodanx2]

Here is how mpv-x86_64-v3-20230903-git-67368ac.7z handles it

[shodanx2]

How WACUP_Preview_v1_0_21_7236.exe handles it Actually it is completely broken, it only shows the first frame then skips to the next time.

Even the original file does not play either. Still I managed to capture the crop factor

[shodanx2]

How smplayer-23.6.0-x64-unsigned.exe handles it This is using the MPV engine

[shodanx2]

How smplayer-23.6.0-x64-unsigned.exe handles it But using the mplayer engine

[shodanx2]

How apprenticevideo-git.replay.20230729.181616.745a82e1-win32-x64.msi handles it

[shodanx2]

Here is how firefox 117.0 (64-bit) handles it

[shodanx2]

How Microsoft Photos 2019.19071.12548.0 handles it

[shodanx2]

How ffplay version 2023-09-04-git-f8503b4c33-full_build-www.gyan.dev handles it

[mifi]

thanks for researching. maybe if we could make a summary/table of each player and how well it works for them, then we could look into whether it's worth implementing

[shodanx2]

Yes, there needs to be a more visual test pattern, make all outputs standard sized and in the same order, so at a glance the difference between players can be seen.

I'm also very curious to know, if I cut up the test pattern video into multiple chunks, set different crop factors for each chunk and then re-assemble them, Will the crop factor change on the fly.

Also all these tests need to be tried with all codecs that support metadata cropping (at least h.265 and AV1) to see whether this feature is getting improvements over new codecs.

Having lossless cropping would be awesome, when I started doing videos it was my second most used action and 3rd was masking.

[Jamim]

Note that, in MKVToolNix GUI, additionally to "Cropping", "Display width/height" must be set to the new video size for distortion-free cropping (should also be possible via "Set aspect ratio").

(This brings us to the idea that Matroska medadata would also make possible the following: aspect ratio correction [spacial stretching/compressing]; adjusting audio delay; temporal stretching/compressing by setting stretch factors and/or FPS per track [even for synced combination of, say, 24-fps video track with English audio track of file 1 and sped-up German audio track of 25-fps video file 2].)

I found that this method works perfectly for YouTube since it respects metadata when transcoding uploaded videos. In my case, I've cut out exactly 108 pixels from the left. By the way, it seems like it's impossible to achieve the same result using h264_metadata.

So @h-h-h-h, thanks a lot!