Offset set before beginning of mdat out of range in Windows

[killervictor]

I have used untrunc_x64 and it maked an error

Info: using 64-bit offsets for the broken file terminate called after throwing an instance of 'std::out_of_range' what(): Offset set before beginning of mdat (-204185700)

Is my broken vedio too large?

[anthwlock]

As your log says, untrunc uses 64bit offsets, so size shouldn't be a problem. However, maybe there are some (int overflow) corner cases. Or maybe it's a symptom of another issue. Can't tell unless I get more info.

So please post the full -v output. Or better yet, send me the files (healthy+broken). I usually recommend wetransfer, but you'd have to split your files to 2GB chunks since that's their limit.

[mandria]

hello, i've the same issue here on windows with the 64bit version here is the -v log i can pm you the files if can help thanks for your help

` Info: version '4f70983' using ffmpeg '3.3.4' Info: reading C:\Users\paoma\Downloads\lamin\C0002.MP4 Info: parsing healthy moov atom ... Composition time offset atom found. Out of order samples possible. Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'C:\Users\paoma\Downloads\lamin\C0002.MP4': Metadata: major_brand : XAVC minor_version : 16785407 compatible_brands: XAVCmp42iso2 creation_time : 2019-12-17T11:22:49.000000Z Duration: 00:07:59.04, start: 0.000000, bitrate: 51687 kb/s Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc), 1920x1080 [SAR 1:1 DAR 16:9], 49945 kb/s, 25 fps, 25 tbr, 25k tbn, 50 tbc (default) Metadata: creation_time : 2019-12-17T11:22:49.000000Z handler_name : Video Media Handler encoder : AVC Coding Stream #0:1(und): Audio: pcm_s16be (twos / 0x736F7774), 48000 Hz, 2 channels, s16, 1536 kb/s (default) Metadata: creation_time : 2019-12-17T11:22:49.000000Z handler_name : Sound Media Handler Stream #0:2(und): Data: none (rtmd / 0x646D7472), 204 kb/s (default) Metadata: creation_time : 2019-12-17T11:22:49.000000Z handler_name : Timed Metadata Media Handler timecode : 14:06:54:11 found avcC after: 102 remaining len:130 parsing avcC ... len_sps: 101 decoding SPS ... log2_max_frame_num: 9 avcC got decoded Info: special track found (meta, 'Timed Metadata Media Handler')

Info: unknown track 'twos' found -> fallback to dynamic stats created dummy track 'free' removed dummy track 'free'

dynamic stats: firstoff: 192 first_offrel: 0 avc1 chunk_distancegcd: 1 likely n_samples/chunk (p=1): 12 likely sample_sizes (p=0): n_mutual_patterns: 0 avc1_twos (0->1) [0] (998) avc1_rtmd (0->2) [0] (0)

twos chunk_distancegcd: 1 likely n_samples/chunk (p=1): 23040 likely sample_sizes (p=1): 4 n_mutual_patterns: 1 twos_rtmd (1->2) [1] (997) 1.000 997 001c0100 00cf0315 f0010010 200e__ twos_avc1 (1->0) [0] (0)

rtmd chunk_distancegcd: 1 likely n_samples/chunk (p=1): 12 likely sample_sizes (p=1): 1024 n_mutual_patterns: 1 rtmd_avc1 (2->0) [1] (998) 1.000 998 00000000 00000000 00000000 00000000 00000002 09100000 00130600 0d80afc8 rtmd_twos (2->1) [0] (0)

Info: using dynamic stats, use '-is' to see them calling findMdat on truncated file.. Info: 'C:\Users\paoma\Downloads\lamin\C0001.MP4' has invalid atom lenghts, see '-f' Info: reading mdat from truncated file ... Info: using 64-bit offsets for the broken file terminate called after throwing an instance of 'std::out_of_range' what(): Offset set before beginning of mdat (-1448836998) `

[anthwlock]

@mandria I think the problem your log reports is fixed in https://github.com/anthwlock/untrunc/commit/3f6562a6c7eac2f45daea3a924e125401125fcc1. However support for rtmd is currently WIP. So I'd expect you get an error like Error: unable to find correct codec -> premature end (~0%). In that case please send me your files (preferably multiple healthy files). I think the first ~50mb (head -c 50M) should be enough. If you get another error, or the same as in your first report, please send the whole file.

[mandria]

sorry but i'm trying on a windows where i cannot build, could you make a new build? i'll send you an email with the access to the files soon

[anthwlock]

win-build

[mandria]

here is the loq with the latest version it seem really similar to the oldest, i've tested with another healthy file

` nfo: version '6581441' using ffmpeg '3.3.4' Info: reading C:\Users\paoma\Downloads\lamin\C0162.MP4 Info: parsing healthy moov atom ... Composition time offset atom found. Out of order samples possible. Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'C:\Users\paoma\Downloads\lamin\C0162.MP4': Metadata: major_brand : XAVC minor_version : 16785407 compatible_brands: XAVCmp42iso2 creation_time : 2019-12-18T13:10:37.000000Z Duration: 00:00:07.68, start: 0.000000, bitrate: 46180 kb/s Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc), 1920x1080 [SAR 1:1 DAR 16:9], 44434 kb/s, 25 fps, 25 tbr, 25k tbn, 50 tbc (default) Metadata: creation_time : 2019-12-18T13:10:37.000000Z handler_name : Video Media Handler encoder : AVC Coding Stream #0:1(und): Audio: pcm_s16be (twos / 0x736F7774), 48000 Hz, 2 channels, s16, 1536 kb/s (default) Metadata: creation_time : 2019-12-18T13:10:37.000000Z handler_name : Sound Media Handler Stream #0:2(und): Data: none (rtmd / 0x646D7472), 204 kb/s (default) Metadata: creation_time : 2019-12-18T13:10:37.000000Z handler_name : Timed Metadata Media Handler timecode : 14:53:18:23 found avcC after: 102 remaining len:130 parsing avcC ... len_sps: 101 decoding SPS ... log2_max_frame_num: 9 avcC got decoded assuming constant duration of 1 for 'twos' (x368640) Info: special track found (meta, 'Timed Metadata Media Handler')

Info: unknown track 'twos' found -> fallback to dynamic stats created dummy track 'free' removed dummy track 'free' ignoring all 4 patterns for 0->1: .. they overlap too much (1.875)

dynamic stats: firstoff: 192 first_offrel: 0 avc1 chunk_distancegcd: 1 likely n_samples/chunk (p=1): 12 likely sample_sizes (p=0): n_mutual_patterns: 0 avc1_twos (0->1) [0] (16) avc1_rtmd (0->2) [0] (0)

twos chunk_distancegcd: 1 likely n_samples/chunk (p=1): 23040 likely sample_sizes (p=1): 4 n_mutual_patterns: 1 twos_rtmd (1->2) [1] (15) 1.000 15 001c0100 00cf0315 f0010010 200e35__ twos_avc1 (1->0) [0] (0)

rtmd chunk_distancegcd: 1 likely n_samples/chunk (p=1): 12 likely sample_sizes (p=1): 1024 n_mutual_patterns: 1 rtmd_avc1 (2->0) [1] (16) 1.000 16 00000000 00000000 00000000 00000000 00000002 09100000 00130600 0d80afc8 rtmd_twos (2->1) [0] (0)

Info: using dynamic stats, use '-is' to see them calling findMdat on truncated file.. Info: 'C:\Users\paoma\Downloads\lamin\C0001.MP4' has invalid atom lenghts, see '-f' Info: reading mdat from truncated file ... Info: using 64-bit offsets for the broken file terminate called after throwing an instance of 'std::out_of_range' what(): Offset set before beginning of mdat (-1448836998) `

[anthwlock]

@mandria It seems like broken Sony XAVC videos have no mp4-strucutre (atoms) at all. This caused untrunc to search (brute-force style) for the 'mdat' atom in the entire file. This is bad because 1. it's not there, 2. the string 'mdat' can occur by chance. This and some other changes later, I was able to recover the file you send me (the 10GB one).

[mandria]

@anthwlock wow great!, i was expecting the file to have atoms, i'll give a try now and let you know

[mandria]

@anthwlock it works like a charm, thanks!

[csabavagyok]

Hello @anthwlock , I am trying to untrunc a dashcam footage, but am a bit stuck. I have successfully recovered part of (?) the video with recover_mp4 and ffmpeg, but the result only shows about 12 minutes, then the video suddenly stops, although it supposedly was still running (?), so I am now trying with untrunc (on Linux, has been working for 3+ hours so far) and your fork of it to try and recover more footage. Unfortunately the GUI version halts on an error: Offset set before beginning of mdat (-353579876). I have tried using -s, --dyn, -k and those combinations, but nothing produces the expected result. The GUI is full of messages like "Warning: New access unit since seen picture (type: 0)", "Warning: Invalid slice type, probably this is not an avc1 sample", there are also some "Warning: Too short!" and "Warning: buffer exceeded by 6123285" messages. I have uploaded the reference file (EMER....MOV) and faulty file (FILE.....MOV) into a shared folder: https://drive.google.com/open?id=1GsIyIz4af4AIFEzfr0EqOIuJtiYwph5E I am hoping that the case is not a malfunctioning camera and to be able to recover footage beyond that 12 minutes and would very much appreciate your help!

[anthwlock]

@csabavagyok After a2c7218b35614 I was able to recover 12min 50sec. I don't think it contains much more. Your healthy file contains ~0.63s/mb. The broken file has ~1.2GB, so you'd expect 0.63 * 1200 = 756s = 12min 36sec.

[csabavagyok]

@anthwlock Thank you very much for your help and effort, this is what I was afraid of :(

[OkamiJ7]

Can you help me? i try with different files, but i cant recover this file using untrunc-gui, here is the whole message but at the final the file result is also corrupted.

Info: parsing healthy moov atom ... Composition time offset atom found. Out of order samples possible.

Info: reading mdat from truncated file ... Info: using 64-bit offsets for the broken file Info: Found 0 packets ( mp4a: 0 mp4a: 0 mp4a: 0 mp4a: 0 mp4a: 0 avc1: 0 avc1-keyframes: 0 ) Info: Duration of mp4a: (0 ms) Info: Duration of mp4a: (0 ms) Info: Duration of mp4a: (0 ms) Info: Duration of mp4a: (0 ms) Info: Duration of mp4a: (0 ms) Info: Duration of avc1: (0 ms) Info: pruned empty 'mp4a' track Info: pruned empty 'mp4a' track Info: pruned empty 'mp4a' track Info: pruned empty 'mp4a' track Info: pruned empty 'mp4a' track Info: pruned empty 'avc1' track Info: saving F:\Youtube\2022-09-28 19-22-36.mp4_fixed.mp4

done!