anthwlock / untrunc

Restore a truncated mp4/mov. Improved version of ponchio/untrunc
GNU General Public License v2.0
1.9k stars 182 forks source link

Sowt is just raw data, no way to guess length #31

Closed mvanderlinde closed 4 years ago

mvanderlinde commented 4 years ago

Getting the following message when attempting to repair a truncated MOV from my dashcam...

Sowt is just raw data, no way to guess length (unless reliably detecting the other codec start)

The process does not error, but produces a file with a duration of 0.

Here is the result of ffmpeg -i for a working file from the same dashcam:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '<redacted>/working.mov': Metadata: major_brand : qt minor_version : 0 compatible_brands: qt creation_time : 2019-06-28T12:24:12.000000Z original_format : NVT-IM original_format-eng: NVT-IM comment : CarDV-TURNKEY comment-eng : CarDV-TURNKEY Duration: 00:03:01.00, start: 0.000000, bitrate: 13078 kb/s Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080, 11879 kb/s, 30 fps, 30 tbr, 30k tbn, 60k tbc (default) Metadata: creation_time : 2019-06-28T12:24:12.000000Z handler_name : VideoHandler encoder : h264 Stream #0:1(eng): Audio: pcm_s16le (sowt / 0x74776F73), 32000 Hz, mono, s16, 512 kb/s (default) Metadata: creation_time : 2019-06-28T12:24:12.000000Z handler_name : SoundHandler

anthwlock commented 4 years ago

I would like to help you, but I need the files you are having problems with. A small sample should be enough, you can use head -c 10M corrupt.mp4 > corrupt_head.mp4. Please send me the link (email or here) to the files and I will take a look.

anthwlock commented 4 years ago

You got my email? I need the working file as well

anthwlock commented 4 years ago

I will close this since the working file is still missing. Once someone has send me a pair of (ok.mp4, bad.mp4) this can be reopened!

DurvalMenezes commented 4 years ago

Hello @anthwlock,

First of all, thanks for your excellent work: your version of untrunc seems much improved over the original!

Second, I'm having the exact same problem as the OP; here's my untrunc -v output as requested:

T:\untrunc_x32>untrunc -v ..\2019_0827_142613_031A.MOV ..\2018_0201_133821_001A.
MOV
Info: version '7364ab1' using ffmpeg '3.3.4'
Info: reading ..\2019_0827_142613_031A.MOV
Info: parsing healthy moov atom ...
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '..\2019_0827_142613_031A.MOV':
  Metadata:
    major_brand     : qt
    minor_version   : 0
    compatible_brands: qt
    creation_time   : 2019-08-27T14:26:48.000000Z
    original_format : NVT-IM
    original_format-eng: NVT-IM
    comment         : CarDV-TURNKEY
    comment-eng     : CarDV-TURNKEY
  Duration: 00:00:36.13, start: 0.000000, bitrate: 13170 kb/s
    Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080
, 11903 kb/s, 30 fps, 30 tbr, 30k tbn, 60k tbc (default)
    Metadata:
      creation_time   : 2019-08-27T14:26:48.000000Z
      handler_name    : DataHandler
      encoder         : h264
    Stream #0:1(eng): Audio: pcm_s16le (sowt / 0x74776F73), 32000 Hz, 1 channels
, s16, 512 kb/s (default)
    Metadata:
      creation_time   : 2019-08-27T14:26:48.000000Z
      handler_name    : DataHandler
found avcC after: 102
remaining len:36
parsing avcC ...
len_sps: 13
decoding SPS ...
log2_max_frame_num: 9
avcC got decoded

calling findMdat on truncated file..
Info: '..\2018_0201_133821_001A.MOV' has invalid atom lenghts, see '-f'
Info: reading mdat from truncated file ...

(reading element from mdat)
Offset: 0 / 16260 : a6efedd7 de6d497b
Track codec: avc1
avc1: no match because of nal type: 30
Track codec: sowt
Sowt is just  raw data, no way to guess length (unless reliably detecting the ot
her codec start)
Error: unable to find correct codec -> premature end (~0%)
       try '-s' to skip unknown sequences

mdat->file_end: 16260
Info: Found 0 packets ( avc1: 0 avc1-keyframes: 0 sowt: 0 )
Info: Duration of avc1:  (0 ms)
Info: Duration of sowt:  (0 ms)
Info: saving ..\2018_0201_133821_001A.MOV_fixed.mp4

T:\untrunc_x32>

Also, here's where you can download the two files mentioned above: http://durval.com/xfer-only/2019_0827_142613_031A.MOV http://durval.com/xfer-only/2018_0201_133821_001A.MOV

BTW, I tried also with the -s option as suggested by the program's output above, with similar results. If you need it, I can try to record and then upload its output here (it was really, really long and seemed mostly uninteresting to me).

TIA, -- Durval.

anthwlock commented 4 years ago

@DurvalMenezes

I added some logic to dynamically (at runtime) detect and use some characteristics from the healthy file. If you want, you can see them all via untrunc -is <good.mov>. Importantly, this enables untrunc to recover whole chunks, rather than just frame by frame. In case of the sowt/twos codec, this is the only way possible.

For such audio codecs, the actual chunk information (found in 'stsc' atom) is needed. So I did something similar to this..

One thing I noticed about your files is that there are hidden data (I think gps) tracks. By default they would get removed, if you want to keep them you can use the '-k' option.

But even then, you would currently lose the content of the (apparantly healthy) 'frea' atom. I have no idea if this data is relevant at all, if you think it is let me know.

With this, I was able to recover the files you uploaded. The code is currently in a seperate branch. If you don't report any bugs/problems, I will soon merge it to master.

~ anthwlock

DurvalMenezes commented 4 years ago

Thanks @anthlock! Will give it a try ASAP!

Problem is, I don't have a build environment for your code, tho... I was using the Win32 binary you made available at https://github.com/anthwlock/untrunc/releases/download/latest/untrunc_x32.zip

Any chance you could release a binary for this separate branch?

TIA, -- Durval.

anthwlock commented 4 years ago

sure thing, they are automated. Look here

DurvalMenezes commented 4 years ago

Thanks again @anthwlock! Downloaded and tested it on the same files as above:

T:\00DAMAGED_VIDEO_FILES>..\untrunc_x32\untrunc.exe ARQUIVO_INTEGRO_-_2019_0827_
142613_031A.MOV 2018_0201_133821_001A.
Info: version '45162f2' using ffmpeg '3.3.4'
Info: reading ARQUIVO_INTEGRO_-_2019_0827_142613_031A.MOV
Info: parsing healthy moov atom ...
Warning: using expected sowt frame size of 1*2, instead of 1 as found in stsz

Info: unknown track 'sowt' found -> fallback to dynamic stats
Info: using dynamic stats, use '-is' to see them
Could not open file: 2018_0201_133821_001A.

..\untrunc_x32\untrunc.exe 2019_0827_
142613_031A.MOV 2018_0201_133821_001A.MOV
Info: version '45162f2' using ffmpeg '3.3.4'
Info: reading 2019_0827_142613_031A.MOV
Info: parsing healthy moov atom ...
Warning: using expected sowt frame size of 1*2, instead of 1 as found in stsz

Info: unknown track 'sowt' found -> fallback to dynamic stats
Info: using dynamic stats, use '-is' to see them
Info: '2018_0201_133821_001A.MOV' has invalid atom lenghts, see '-f'
Info: reading mdat from truncated file ...
Info: beginning at offset 245884 instead of 0
Error: unable to find correct codec -> premature end (~35.58%)
       try '-s' to skip unknown sequences

Info: Found 99196 packets ( avc1: 60 avc1-keyframes: 6 sowt: 32000 )
Tip: Audio and video seem to have different durations (0.5).
     If audio and video are not in sync, give `-sv` a try. See `--help`
Info: Duration of avc1: 2s  (2000 ms)
Info: Duration of sowt: 1s  (1000 ms)
Warning: Unknown sequences: 3
Warning: Bytes NOT matched: 306KiB (6.703%)
Info: saving 2018_0201_133821_001A.MOV_fixed-dyn.MOV

Warning: 59 warnings were hidden!

The resulting file looks good, less than 2 secs of video but the original (damaged) file was quite small too, so probably not much was lost in the recovery.

But then I tested on a much larger (and also damaged) file I had around: 09/09/2018 10:31 PM 233,693,184 2018_0908_101613_003A.MOV

Here's the output for the untrunc execution:

T:\00DAMAGED_VIDEO_FILES>..\untrunc_x32\untrunc.exe 2019_0827_
142613_031A.MOV 2018_0908_101613_003A.MOV
Info: version '45162f2' using ffmpeg '3.3.4'
Info: reading 2019_0827_142613_031A.MOV
Info: parsing healthy moov atom ...
Warning: using expected sowt frame size of 1*2, instead of 1 as found in stsz

Info: unknown track 'sowt' found -> fallback to dynamic stats
Info: using dynamic stats, use '-is' to see them
Info: '2018_0908_101613_003A.MOV' has invalid atom lenghts, see '-f'
Info: reading mdat from truncated file ...
Info: beginning at offset 35996 instead of 0
Error: unable to find correct codec -> premature end (~1.377%)
       try '-s' to skip unknown sequences

Info: Found 99196 packets ( avc1: 60 avc1-keyframes: 6 sowt: 32000 )
Tip: Audio and video seem to have different durations (0.5).
     If audio and video are not in sync, give `-sv` a try. See `--help`
Info: Duration of avc1: 2s  (2000 ms)
Info: Duration of sowt: 1s  (1000 ms)
Warning: Unknown sequences: 3
Warning: Bytes NOT matched: 101KiB (3.208%)
Info: saving 2018_0908_101613_003A.MOV_fixed-dyn.MOV

Warning: 59 warnings were hidden!

And here's the recovered file: 09/23/2019 07:54 PM 3,112,821 2018_0908_101613_003A.MOV_fixed-dyn.MOV

So, from a 233MB file, only 3MB resulted... as expected, less than 2 secs again on the recovered file (and projecting from its size, the damaged file should have had at least 2 minutes of video on it).

Can something be done regarding that?

Here's the output with -v:

Info: version '45162f2' using ffmpeg '3.3.4'
Info: reading 2019_0827_142613_031A.MOV
Info: parsing healthy moov atom ...
found avcC after: 102
remaining len:36
parsing avcC ...
len_sps: 13
decoding SPS ...
log2_max_frame_num: 9
avcC got decoded
Warning: using expected sowt frame size of 1*2, instead of 1 as found in stsz

Info: unknown track 'sowt' found -> fallback to dynamic stats
created dummy track 'free'

dynamic stats:
first_off_: 262144
first_off_rel_: 134488
avc1
chunk_distance_gcd_: 32768
likely n_samples/chunk (p=0.972973): 30
likely sample_sizes (p=0):
n_mutual_patterns: 1
avc1_free (0->2) [1] (37)
1.000 37  ________ ________ ________ ________ ________ ________ ________ _______
_ 0000____ 66726565 00000000 00000000 00000000 00000000 00000000 00000000
avc1_sowt (0->1) [0] (0)

sowt
chunk_distance_gcd_: 32768
likely n_samples/chunk (p=0.972973): 32000
likely sample_sizes (p=1): 2
n_mutual_patterns: 1
sowt_free (1->2) [1] (36)
1.000 36  ________ ________ ________ ________ ________ ________ ________ _______
_ 00000600 66726565 00000000 00000000 00000000 00000000 00000000 00000000
sowt_avc1 (1->0) [0] (0)

free
chunk_distance_gcd_: 4
likely n_samples/chunk (p=0.493151): 67072
likely sample_sizes (p=1): 1
n_mutual_patterns: 2
free_avc1 (2->0) [1] (36)
1.000 36  00000000 00000000 00000000 00000000 00000000 00000000 00800000 636c757
2 00______ 65888000 00______ ________ ________ ________ ________ ________
free_sowt (2->1) [1] (37)
1.000 37  00000000 00000000 00000000 00000000 00000000 00000000 00000000 0000000
0 ________ ________ ________ ________ ________ ________ ________ ________

Info: using dynamic stats, use '-is' to see them
calling findMdat on truncated file..
Info: '2018_0908_101613_003A.MOV' has invalid atom lenghts, see '-f'
Info: reading mdat from truncated file ...
avc1: no match because of nal type: 31
wouldMatchDyn(0, -1) -> no
wouldMatch(0, "", 0) -> no
avc1: Match with 0 header
wouldMatch(35996, "", 0) -> yes, avc1
Info: beginning at offset 35996 instead of 0

(reading element from mdat)
Offset: 35996 / 262144 : 00035100 65888000
Track codec: avc1
avc1: Match with 0 header
sps_info (before): 1 4 5 0
sps_info (after):  0 9 6 0
---
Length: 217348
Ref idc: 3
Nal type: 5
Partial avc1-length: 217348
---
Length: 33204
Ref idc: 2
Nal type: 1
Warning: Different nal type (5, 1)
part-length: 217348
1th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 253344 / 479492 : 000081b0 419a0002
Track codec: avc1
avc1: Match with 0 header
---
Length: 33204
Ref idc: 2
Nal type: 1
Partial avc1-length: 33204
---
Length: 34616
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 33204
2th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 286548 / 512696 : 00008734 419a0004
Track codec: avc1
avc1: Match with 0 header
---
Length: 34616
Ref idc: 2
Nal type: 1
Partial avc1-length: 34616
---
Length: 21236
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 34616
3th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 321164 / 547312 : 000052f0 419a0006
Track codec: avc1
avc1: Match with 0 header
---
Length: 21236
Ref idc: 2
Nal type: 1
Partial avc1-length: 21236
---
Length: 39656
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 21236
4th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 342400 / 568548 : 00009ae4 419a0008
Track codec: avc1
avc1: Match with 0 header
---
Length: 39656
Ref idc: 2
Nal type: 1
Partial avc1-length: 39656
---
Length: 47488
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 39656
5th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 382056 / 608204 : 0000b97c 419a000a
Track codec: avc1
avc1: Match with 0 header
---
Length: 47488
Ref idc: 2
Nal type: 1
Partial avc1-length: 47488
---
Length: 27532
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 47488
6th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 429544 / 655692 : 00006b88 419a000c
Track codec: avc1
avc1: Match with 0 header
---
Length: 27532
Ref idc: 2
Nal type: 1
Partial avc1-length: 27532
---
Length: 51224
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 27532
7th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 457076 / 683224 : 0000c814 419a000e
Track codec: avc1
avc1: Match with 0 header
---
Length: 51224
Ref idc: 2
Nal type: 1
Partial avc1-length: 51224
---
Length: 30544
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 51224
8th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 508300 / 734448 : 0000774c 419a0011
Track codec: avc1
avc1: Match with 0 header
---
Length: 30544
Ref idc: 2
Nal type: 1
Partial avc1-length: 30544
---
Length: 49696
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 30544
9th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 538844 / 764992 : 0000c21c 419a0013
Track codec: avc1
avc1: Match with 0 header
---
Length: 49696
Ref idc: 2
Nal type: 1
Partial avc1-length: 49696
---
Length: 26004
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 49696
10th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 588540 / 814688 : 00006590 419a0015
Track codec: avc1
avc1: Match with 0 header
---
Length: 26004
Ref idc: 2
Nal type: 1
Partial avc1-length: 26004
---
Length: 49016
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 26004
11th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 614544 / 840692 : 0000bf74 419a0017
Track codec: avc1
avc1: Match with 0 header
---
Length: 49016
Ref idc: 2
Nal type: 1
Partial avc1-length: 49016
---
Length: 32768
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 49016
12th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 663560 / 889708 : 00007ffc 419a0019
Track codec: avc1
avc1: Match with 0 header
---
Length: 32768
Ref idc: 2
Nal type: 1
Partial avc1-length: 32768
---
Length: 70508
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 32768
13th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 696328 / 922476 : 00011368 419a001b
Track codec: avc1
avc1: Match with 0 header
---
Length: 70508
Ref idc: 2
Nal type: 1
Partial avc1-length: 70508
---
Length: 13844
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 70508
14th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 766836 / 992984 : 00003610 419a001d
Track codec: avc1
avc1: Match with 0 header
---
Length: 13844
Ref idc: 2
Nal type: 1
Partial avc1-length: 13844
---
Length: 217228
Ref idc: 3
Nal type: 5
Warning: Different nal type (5, 1)
part-length: 13844
15th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 780680 / 1006828 : 00035088 65888000
Track codec: avc1
avc1: Match with 0 header
---
Length: 217228
Ref idc: 3
Nal type: 5
Partial avc1-length: 217228
---
Length: 39708
Ref idc: 2
Nal type: 1
Warning: Different nal type (5, 1)
part-length: 217228
16th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 997908 / 1224056 : 00009b18 419a0002
Track codec: avc1
avc1: Match with 0 header
---
Length: 39708
Ref idc: 2
Nal type: 1
Partial avc1-length: 39708
---
Length: 19668
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 39708
17th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 1037616 / 1263764 : 00004cd0 419a0004
Track codec: avc1
avc1: Match with 0 header
---
Length: 19668
Ref idc: 2
Nal type: 1
Partial avc1-length: 19668
---
Length: 46048
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 19668
18th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 1057284 / 1283432 : 0000b3dc 419a0006
Track codec: avc1
avc1: Match with 0 header
---
Length: 46048
Ref idc: 2
Nal type: 1
Partial avc1-length: 46048
---
Length: 31116
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 46048
19th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 1103332 / 1329480 : 00007988 419a0008
Track codec: avc1
avc1: Match with 0 header
---
Length: 31116
Ref idc: 2
Nal type: 1
Partial avc1-length: 31116
---
Length: 46752
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 31116
20th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 1134448 / 1360596 : 0000b69c 419a000a
Track codec: avc1
avc1: Match with 0 header
---
Length: 46752
Ref idc: 2
Nal type: 1
Partial avc1-length: 46752
---
Length: 31044
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 46752
21th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 1181200 / 1407348 : 00007940 419a000c
Track codec: avc1
avc1: Match with 0 header
---
Length: 31044
Ref idc: 2
Nal type: 1
Partial avc1-length: 31044
---
Length: 55460
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 31044
22th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 1212244 / 1438392 : 0000d8a0 419a000e
Track codec: avc1
avc1: Match with 0 header
---
Length: 55460
Ref idc: 2
Nal type: 1
Partial avc1-length: 55460
---
Length: 28724
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 55460
23th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 1267704 / 1493852 : 00007030 419a0011
Track codec: avc1
avc1: Match with 0 header
---
Length: 28724
Ref idc: 2
Nal type: 1
Partial avc1-length: 28724
---
Length: 54668
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 28724
24th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 1296428 / 1522576 : 0000d588 419a0013
Track codec: avc1
avc1: Match with 0 header
---
Length: 54668
Ref idc: 2
Nal type: 1
Partial avc1-length: 54668
---
Length: 20492
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 54668
25th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 1351096 / 1577244 : 00005008 419a0015
Track codec: avc1
avc1: Match with 0 header
---
Length: 20492
Ref idc: 2
Nal type: 1
Partial avc1-length: 20492
---
Length: 25528
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 20492
26th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 1371588 / 1597736 : 000063b4 419a0017
Track codec: avc1
avc1: Match with 0 header
---
Length: 25528
Ref idc: 2
Nal type: 1
Partial avc1-length: 25528
---
Length: 96684
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 25528
27th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 1397116 / 1623264 : 000179a8 419a0019
Track codec: avc1
avc1: Match with 0 header
---
Length: 96684
Ref idc: 2
Nal type: 1
Partial avc1-length: 96684
---
Length: 24068
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 96684
28th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 1493800 / 1719948 : 00005e00 419a001b
Track codec: avc1
avc1: Match with 0 header
---
Length: 24068
Ref idc: 2
Nal type: 1
Partial avc1-length: 24068
---
Length: 63464
Ref idc: 2
Nal type: 1
Warning: Different poc lsb
part-length: 24068
29th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 1517868 / 1744016 : 0000f7e4 419a001d
Track codec: avc1
avc1: Match with 0 header
---
Length: 63464
Ref idc: 2
Nal type: 1
Partial avc1-length: 63464
---
Length: 27532
Ref idc: 3
Nal type: 6
Partial avc1-length: 90996
---
First byte expected 0
failed parsing nal-header
part-length: 90996
30th sample in 1th avc1-chunk

(reading element from mdat)
Offset: 1608864 / 1835012 : b0022002 dc011002
trying chunkPredict first..
called getChunkPrediction(1608864) ...
no chunk with future found
Track codec: avc1
avc1: no match because of nal type: 28
Track codec: sowt
offset not accepted as start-of-chunk offset by 'sowt'
Track codec: free
Error: unable to find correct codec -> premature end (~0.6891%)
       try '-s' to skip unknown sequences

mdat->file_end: 1835012
Info: Found 30 packets ( avc1: 30 avc1-keyframes: 3 sowt: 0 )
Info: Duration of avc1: 1s  (1000 ms)
Info: Duration of sowt:  (0 ms)
Warning: Unknown sequences: 1
Warning: Bytes NOT matched: 35.2KiB (2.237%)
Info: pruned empty 'sowt' track
Info: saving 2018_0908_101613_003A.MOV_fixed-dyn.MOV
skipped 0 35996

If you need the last damaged file above, please let me know and I will try to upload it somewhere (I have really bad connectivity right now).

TIA, -- Durval.

anthwlock commented 4 years ago

Perhaps it's working better now?

I was able to recover 5s for 2018_0201_133821_001A.MOV. If you are able to do the same, but still have problems with the bigger one, I will need the file to see where the problem is.

anthwlock commented 4 years ago

@DurvalMenezes If you encounter any more problems let me know.

DurvalMenezes commented 4 years ago

Sorry for the delay in responding. @anthlock -- went on a trip to a place without internet access for the last 2 weeks.

I've just returned to civilization and was able to test the new binary you posted above.

The results are definitely better: of the 8 damaged video files I have here, in 5 of them it was able to recover ~90% or more (judging by file size) of the original content. But in the other 3, it lost a substantial amounts, ~95% or more. In the worst one, only ~275KB were recovered from a ~212MB file... :-/

I will be uploading this specific file (called 2018_1207_143841_032A.MOV) to a different location, and then I will send you the link privately (direct to your email address), as I do not know what could be in that video and it could well be something private... ;-)

Thanks again for your hard work in fixing this.

anthwlock commented 4 years ago

@DurvalMenezes I think it actually recovers 99% of the video/audio, the rest is embedded user/gps data. If you think you will need this data later, you can use -k to keep it.

After the last two commits I was able to repair the file you send me. Hope the rest works as well!

DurvalMenezes commented 4 years ago

Great job @anthwlock!

By "the last two commits", I think you mean this specific version, and from that I was able to deduce this path to the binary; if I'm wrong on any of that, please set me straight.

Presuming it's the correct version, I just downloaded and tested it, seems we are almost there:

I've just uploaded these two files and sent you the links privately to your email address; when you have the leisure, please have a look at them and see what more can be done to untrunc to allow it to recover them.

Thanks again for your continued attention, -- Durval.

anthwlock commented 4 years ago

The latest commits always builds upon the previous ones, so the one you chose "includes" the other one, so you did everything right.

One problem was that chunks starting with zeros were skipped. Another problem was that the zero padding (at end of 'free' atom) was smaller than in the reference file, fixed that by reducing Mp4::pat_size_ to 32.

I realized that those 'free' chunks actually consist out of 'free' atoms, so their size can be determined exactly.

~ anthwlock

DurvalMenezes commented 4 years ago

Hello @anthwlock,

Thanks for the info on versioning and for the confirmation I'm using the right version and binary.

Thanks specially for hunting down those two additional problems; just checked untrunc's commit log and I deduce they were addressed by be7936b and ff2f41e, which should both be in the latest master (8248fad), so I downloaded its binary from here.

I then tested it with the recalcitrant files 2018_0621_195705_008A.MOV and 2018_0908_095613_001A.MOV I mentioned above; By the volume of bytes, this new version of yours recovered ~94.9% and 94.8% respectively, so I think we can call it a success, and claim the problem as solved.

Your hard work on untrunc has really helped me, so I sent a little donation your way, I hope you enjoy it.

Thanks again, -- Durval.

anthwlock commented 4 years ago

Thank you very much for the donation!