Warning: addToExclude: sequence goes beyond EOF

[kesaroid]

I was trying to view a video recorded on a Basler camera with an MP4 container. ffmpeg gave me the error [mov,mp4,m4a,3gp,3g2,mj2 @ 000001572b9fa3a0] moov atom not found MVI_3891.MOV: Invalid data found when processing input

Upon using -s argument, I get a great number of Warning: addToExclude: sequence goes beyond EOF along with Warning: Skipping minf atom: 3633354601

Even though it says duration of video 11 minutes, it crashes after 40 seconds. Do you know what the reason could be? Here are the last few lines of the log

Warning: NOT skipping m4ds atom: 3603990810 (at 13989369272 / 13989369316)    
Warning: addToExclude: sequence goes beyond EOF
Warning: addToExclude: sequence goes beyond EOF
Warning: addToExclude: sequence goes beyond EOF
Info: Found 13907 packets ( mp4v: 13907 )
Info: Duration of mp4v: 11min 35s 350ms  (695350 ms)
Warning: Unknown sequences: 2107
Warning: Bytes NOT matched: 498MiB (3.732%)
Info: saving Video.mp4
Assertion failed!
Program: F:\Program Files\untrunc_x64\untrunc.exe
File: src/atom.cpp, Line 481
Expression: end - start == new_length

[anthwlock]

Do you have enough space on your disk? Does the assertion still fail when using -k? Could you upload the entire log somewhere? (either as attachment or on a pastebin)

Your file seems to be big (14-17GB ?), and when using -s, it can happen that (by chance) a random 4-byte sequence is detected as one of the ~180 atoms. untrunc then falsely skips this "atom".

Even though it says duration of video 11 minutes, it crashes after 40 seconds

What do you mean? Does your video-player crash after 40s, or has the resulting file only 40 seconds in total?

As always, having access to the files (healthy+broken) would help the debugging process, and might even be necessary for any progress. I think https://mega.nz/ allows 50GB files for new (free) accounts. The limit gets reduced to 15GB after 1month though.

[Remboooo]

I think I'm running into the same problem. The source MP4 is from a Sony 4k camera (AX-53). I'm also seeing Warning: addToExclude: sequence goes beyond EOF followed by the eventual assertion failure on line 481 of atom.cpp.

I know some C++, but have no experience with MP4 or any video formats for that matter, so I really hope someone can shed some light on this.

My source file can be found here (warning: 26.5GB!). The intact reference file I use is here.

When I run untrunc -s -vv C0010.MP4 C0011.MP4 I get a C0011.MP4_fixed-s1-dyn.MP4 that is only 1,21GB before crashing on that assertion. I have both the full -vv log (warning: 68MB of text) and the normal log.

The short version of the log is:

WARNING: Because of internal decoder changes, using ffmpeg '4.2.4-1ubuntu0.1' with '-s' can be slow!
         You are advised to compile untrunc against ffmpeg 3.3!
         See the README.md on how to do that.
Info: version '8024f0c' using ffmpeg '4.2.4-1ubuntu0.1'
Info: reading C0010.MP4
Info: parsing healthy moov atom ...
Composition time offset atom found. Out of order samples possible.
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x561a2b770dc0] st: 0 edit list: 1 Missing key frame while searching for timestamp: 1000
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x561a2b770dc0] st: 0 edit list 1 Cannot find an index entry before timestamp: 1000.
Info: special track found (meta, 'Timed Metadata Media Handler')

Info: unknown track 'twos' found -> fallback to dynamic stats
Info: using dynamic stats, use '-is' to see them
Warning: no mp4-structure found in: 'C0011.MP4'
Info: using orig_mdat_start_ (=176)
Info: reading mdat from truncated file ...
Info: using 64-bit offsets for the broken file
Warning: Skipping devc atom: 3742891039
Warning: addToExclude: sequence goes beyond EOF
Warning: addToExclude: sequence goes beyond EOF
Warning: addToExclude: sequence goes beyond EOF
Warning: addToExclude: sequence goes beyond EOF
Warning: Invalid slice type, probably this is not an avc1 sample
Warning: addToExclude: sequence goes beyond EOF
Info: Found 92832610 packets ( avc1: 48286 avc1-keyframes: 2013 twos: 92736000 rtmd: 48324 )
Info: Duration of avc1: 32min 11s 440ms  (1931440 ms)
Info: Duration of twos: 32min 12s  (1932000 ms)
Info: Duration of rtmd: 32min 12s 960ms  (1932960 ms)
Warning: Unknown sequences: 6
Warning: Bytes NOT matched: 2.1MiB (0.008115%)
Info: saving C0011.MP4_fixed-s1-dyn.MP4
untrunc: src/atom.cpp:481: void BufferedAtom::write(FileWrite&, bool): Assertion `end - start == new_length' failed.

For completeness I also ran untrunc -vv C0010.MP4 C0011.MP4 (so without -s), which gives me C0011.MP4_fixed-dyn.MP4 of almost the same size (507kiB less) but with the assertion failure replaced by 'unable to find correct codec, try -s to skip unknown sequences':

Offset: 1301221545 / 1301221729 : 12fb4304 6f91d545
trying chunkPredict first.. 
called getChunkPrediction(1301221545 / 1301221729) ... 
Track codec: avc1
avc1: failed for no particular reason
Track codec: twos
Track codec: rtmd
Error: unable to find correct codec -> premature end (~4.788%)
       try '-s' to skip unknown sequences

mdat->file_end: 1301221729
Info: Found 5189401 packets ( avc1: 2689 avc1-keyframes: 113 twos: 5184000 rtmd: 2712 )
first_failed: 0 of 544
order: (1, 3000) (2, 0) 
reduced origt_ctts_: 544 -> 2
Info: Duration of avc1: 1min 47s 560ms  (107560 ms)
Info: Duration of twos: 1min 48s  (108000 ms)
Info: Duration of rtmd: 1min 48s 480ms  (108480 ms)
Warning: Unknown sequences: 1
Warning: Bytes NOT matched: 8B (6.148e-07%)
Info: saving C0011.MP4_fixed-dyn.MP4
skipping 8      at 0 / 184
size_requested: 15728640

Full -vv log here.

[anthwlock]

This should be fixed with d4f2db7801ec11cc04c30a2310101b390970fc4b. I tested it with @Remboooo's files, and the result (using -s) now has a duration of 26min 49s.