Examples on how to parse MKV files

[greatwallet]

Is your feature request related to a problem? Please describe.

I am a green hand upon video processing and using ffmpeg or mkvtoolnix. And there is almost no example codes in documentation, except one or two lines of commands in "https://docs.microsoft.com/en-us/azure/kinect-dk/record-file-format", which only gives commands on getting depth image via ffmpeg, so it's really painful for me to get stuck into video processing.

Describe the solution you'd like

Would you plz give some example commands of ffmpeg and mkvextract(mkvmerge or something alike) to extract rgb, depth, ir, imu meta and all sorts of data modalities?

Describe alternatives you've considered

Maybe in the future, would you work on giving functions on k4aviewer so that users can extract any frame or data of any modality on any timestamp via a simple button press?

Additional context

Currently I need to extract a huge amount of mkv files and extracts rgb, ir and depth frames, so I want some exact commands so that I can write cmd files for batch processing. Currently I could not get ir frames when I tried

ffmpeg -i .\test.mkv -map 0:3 -vsync 0 -vframes 20 test\\ir_test_%04d.png

, and error reported Automatic encoder selection failed for output stream #0:0. Default encoder for format image2 (codec none) is probably disabled. Please choose an encoder manually. Error selecting an encoder for stream 0:0 while the flag for map works for depth and rgb when I tried 0:1 and 0:0 respectively.

If anybody gets a clue on that, plz help me out.

[RoseFlunder]

If I remember correctly you could get the IR images with: ffmpeg -i test.mkv -map 0:2 -vsync 0 ir%04d.png

Depth images: ffmpeg -i test.mkv -map 0:1 -vsync 0 depth%04d.png

RGB Images: ffmpeg -i test.mkv -map 0:0 -vsync 0 color%04d.png

But I am no expert either. Just used these commands once.

[vladalx]

There is an example in the documentation here https://docs.microsoft.com/en-us/azure/kinect-dk/record-file-format

for IR you should use something like ffmpeg -i test.mkv -map 0:2 -vsync 0 -frames 20 ir%04d.png

[greatwallet]

Got it! Thx everyone!

[jhljx]

So how to extract the IMU data from the mkv file by a ffmpeg command?

ffmpeg -i test.mkv -map 0:3 -vsync 0 imu%04d.txt doesn't work. The errors are

Unknown/unsupported AVCodecID S_K4A/IMU.
[matroska,webm @ 00000189ba64d6c0] Could not find codec parameters for stream 3 (Subtitle: none): unknown codec
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[matroska,webm @ 00000189ba64d6c0] Could not find codec parameters for stream 4 (Attachment: none): unknown codec
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options

It says that the IMU data has an unknown codec.

[ziangcao0312]

I have the same question too.

[aiwhoo]

@RoseFlunder , how would you take the inverse of your ffmpeg call? So instead of breaking the mkv into the three channels, how can we provide the 3 channels (ie ir%04d.png depth%04d.png and color%04d.png in png format and create the mkv?

[mrcfschr]

Can we reopen this issue? The documentation mentions Tools such as ffmpeg or the mkvinfo command from the MKVToolNix toolkit can be used to view and extract information from recording files. Besides ffmpeg I also tried MKVToolNix: C:\Program Files\MKVToolNix>mkvextract.exe D:\Desktop\output.mkv tracks 0:color.mkv 1:depth.mkv 2:ir.mkv 3:imu.mkv with the result: Error: Extraction of track ID 3 with the CodecID 'S_K4A/IMU' is not supported. All other mkv content can be parsed, it's only IMU I'm having trouble with using both ffmpeg as already described above and mkvextract. Would be interesting to add examples for re-merging tracks to the documentation as well.

[Marselka]

I faced the same problem as @marcfischer123467 and other colleagues, and didn't find a solution on how to extract IMU data from mkv. Could one from the DK developers prompt us?

[cansik]

To extract the calibration data it is possible to use the following ffmpeg command:

ffmpeg -dump_attachment:t "" -i OFFICE_SAMPLE_NFOV_2X2BINNED.mkv

[lijuno]

mkvextract output.mkv tracks --raw 3:imu.bin dumped IMU data from "output.mkv".

Ref: https://mkvtoolnix.download/doc/mkvextract.html#mkvextract.description.tracks

[ashutosh3308]

How to merge back the extracted frames to a video so that it can be played using Azure Kinect Viewer?

[nosignal00]

For whom still find the way with ffmpeg ffmpeg -i inputfile -map 0:3 -c copy -f data IMU.data

You can open the file with hex editor. The binary format is here. https://learn.microsoft.com/en-us/azure/kinect-dk/record-file-format#imu-sample-structure