search

anthwlock / untrunc

Restore a truncated mp4/mov. Improved version of ponchio/untrunc
GNU General Public License v2.0
1.9k stars 182 forks source link

unable to find image locally #44

Closed broncobenshea closed 4 years ago

broncobenshea commented 4 years ago

Hey Anthwlock,

getting some strange language. Any ideas?

Benjamins-MBP:untrunc-latest bronco$ docker run -v ~/Videos/:/mnt untrunc ~/Downloads/Videos/C0003.mp4 ~/Downloads/Videos/C0002.mp4 Info: version '' using ffmpeg '3.4.6-0ubuntu0.18.04.1' Info: reading /Users/bronco/Downloads/Videos/C0003.mp4 Could not open file: /Users/bronco/Downloads/Videos/C0003.mp4

broncobenshea commented 4 years ago

could it be that 3.4.6 of FFmpeg doesn't play nice?

anthwlock commented 4 years ago

As you might know, docker runs everything in a container for you, separated from the host. In particular files are separated. That's why it can't open your files. Luckily there is an -v option for docker run, with let's you mount any host directory to specified container directory. In your case: docker run -v ~/Downloads/Videos/:/mnt untrunc /mnt/C0003.mp4 /mnt/C0002.mp4

broncobenshea commented 4 years ago

I appreciate you! that code worked. If I get this error do you think all is lost? enjamins-MBP:untrunc-latest bronco$ docker run -v ~/Downloads/Videos/:/mnt untrunc /mnt/C0003.mp4 /mnt/C0002.mp4 Info: version '' using ffmpeg '3.4.6-0ubuntu0.18.04.1' Info: reading /mnt/C0003.mp4 Info: parsing healthy moov atom ... Composition time offset atom found. Out of order samples possible. Info: avcC got decoded Could not open file: /mnt/C0002.mp4

anthwlock commented 4 years ago

What does ls -l ~/Downloads/Videos/C0002.mp4 output?

broncobenshea commented 4 years ago

Benjamins-MBP:untrunc-latest bronco$ ls -l /Users/bronco/Downloads/Videos/C0002.mov -rwxrwxrwx@ 1 bronco staff 16325088150 Mar 20 18:57 /Users/bronco/Downloads/Videos/C0002.mov

broncobenshea commented 4 years ago

disregard. I see the file error. will report back

broncobenshea commented 4 years ago

Benjamins-MBP:untrunc-latest bronco$ docker run -v ~/Downloads/Videos/:/mnt untrunc /mnt/C0003.mp4 /mnt/C0002.mp4 Info: version '' using ffmpeg '3.4.6-0ubuntu0.18.04.1' Info: reading /mnt/C0003.mp4 Info: parsing healthy moov atom ... Composition time offset atom found. Out of order samples possible. Info: avcC got decoded Info: searching start of mdat ... invalid atom name: 'drm '

broncobenshea commented 4 years ago

it got to 1.1% and stopped

anthwlock commented 4 years ago

It looks like you are using an outdated version of untrunc. Please recompile and try again.

broncobenshea commented 4 years ago

ok, I just re downloaded the zip package from your hub. I'll try again.

Thanks very much for your help. Donating you some beers after this.

broncobenshea commented 4 years ago

still failed. I'm downloading untrunc latest from your releases section. then using the readme to compile with docker.

crashes at 1.1%.

anthwlock commented 4 years ago

oh wow, I just checked and the .zip in that release section is still from may 2019. The .exe one can find there is up to date though. Will take a look later if it's possible to either remove the .zip or update it (automatically).

For now, please download from https://github.com/anthwlock/untrunc/archive/master.zip.

broncobenshea commented 4 years ago

thanks for that update! here is the output Benjamins-MBP:untrunc-master bronco$ docker run -v ~/Downloads/Videos/:/mnt untrunc /mnt/C0003.mp4 /mnt/C0002.mp4 Composition time offset atom found. Out of order samples possible. Info: version '' using ffmpeg '3.4.6-0ubuntu0.18.04.1' Info: reading /mnt/C0003.mp4 Info: parsing healthy moov atom ...

Info: unknown track 'twos' found -> fallback to dynamic stats Info: using dynamic stats, use '-is' to see them Warning: no mp4-structure found in: '/mnt/C0002.mp4' Info: assuming start_offset=0. use '-sm' to search for 'mdat' atom instead (via brute-force) Info: reading mdat from truncated file ... Info: using 64-bit offsets for the broken file orig_trak:twos 1 0 == 3 no pattern suggests this.. Error: unable to find correct codec -> premature end (~0%) try '-s' to skip unknown sequences

Info: Found 0 packets ( avc1: 0 avc1-keyframes: 0 twos: 0 ) Info: Duration of avc1: (0 ms) Info: Duration of twos: (0 ms) Info: pruned empty 'avc1' track Info: pruned empty 'twos' track Info: saving /mnt/C0002.mp4_fixed-dyn.mp4 Benjamins-MBP:untrunc-master bronco$

file output is only 160kb

anthwlock commented 4 years ago

Can you send me the files? If they are too big, you can now use untrunc -sh file.mp4 to extract the first ~200MB. If you send me a log, please use the -v option, in your case like this: docker run -v ~/Downloads/Videos/:/mnt untrunc -v /mnt/C0003.mp4 /mnt/C0002.mp4

broncobenshea commented 4 years ago

so the corrupted file is ~17GB so I don't think I can send. The untrunc -sh said command not found.

here is what the -v log output

Benjamins-MBP:untrunc-master bronco$ docker run -v ~/Downloads/Videos/:/mnt untrunc -v /mnt/C0003.mp4 /mnt/C0002.mp4 Info: version '' using ffmpeg '3.4.6-0ubuntu0.18.04.1' Info: reading /mnt/C0003.mp4 Info: parsing healthy moov atom ... Composition time offset atom found. Out of order samples possible. Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/mnt/C0003.mp4': Metadata: major_brand : mp42 minor_version : 1 compatible_brands: isommp41mp42 creation_time : 2020-03-21T19:55:30.000000Z Duration: 00:01:01.88, start: 0.000000, bitrate: 96442 kb/s Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709/bt709/iec61966-2-4), 3840x2160 [SAR 1:1 DAR 16:9], 94826 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default) Metadata: creation_time : 2020-03-21T19:55:30.000000Z handler_name : Core Media Video encoder : AVC Coding Stream #0:1(und): Audio: pcm_s16be (twos / 0x736F7774), 48000 Hz, 2 channels, s16, 1536 kb/s (default) Metadata: creation_time : 2020-03-21T19:55:30.000000Z handlername : Core Media Audio ftyp = 'mp42' found avcC after: 102 remaining len:421 parsing avcC ... len_sps: 52 decoding SPS ... log2_max_frame_num: 6 avcC got decoded assuming constant duration of 1001 for 'avc1' (x1856) assuming constant duration of 1 for 'twos' (x2970184)

Info: unknown track 'twos' found -> fallback to dynamic stats created dummy track 'free' removed dummy track 'free' first_failed: 2 of 244 order_finished: 1 order: (1, 48000) isTrackOrderEnough: 0 (sz=0) ignoring all 13 patterns for 0->1: .. they overlap too much (5.42)

dynamic stats: firstoff: 44 first_offrel: 0 avc1 chunk_distancegcd: 1 likely n_samples/chunk (p=0.983471): 15 likely sample_sizes (p=0): n_mutual_patterns: 0 avc1_twos (0->1) [0] (120)

twos chunk_distancegcd: 1 likely n_samples/chunk (p=0.96748): 24024 likely sample_sizes (p=1): 4 n_mutual_patterns: 2 twos_avc1 (1->0) [1] (121) 0.826 100 ____ ____ ____ ____ 00000002 090000 0006 __

Info: using dynamic stats, use '-is' to see them calling findMdat on truncated file.. Warning: no mp4-structure found in: '/mnt/C0002.mp4' Info: assuming start_offset=0. use '-sm' to search for 'mdat' atom instead (via brute-force) Info: reading mdat from truncated file ... Info: using 64-bit offsets for the broken file avc1: no match because of nal type: 22 avc1: failed for no particular reason

(reading element from mdat) Offset: 0 / 0 : a57384db 130f5683 trying chunkPredict first.. called getChunkPrediction(0 / 0) ... orig_trak:twos 1 a57384db 130f5683 b63907ed 2f26baf4 b5c67879 7c24ac53 c692c73d 59ab5dbe ____ ____ ff__ 00__ 00__ ____ ____ ____ 0 == 3 no pattern suggests this.. Track codec: avc1 avc1: failed for no particular reason Track codec: twos Error: unable to find correct codec -> premature end (~0%) try '-s' to skip unknown sequences

mdat->file_end: 0 Info: Found 0 packets ( avc1: 0 avc1-keyframes: 0 twos: 0 ) Info: Duration of avc1: (0 ms) Info: Duration of twos: (0 ms) Info: pruned empty 'avc1' track Info: pruned empty 'twos' track Info: saving /mnt/C0002.mp4_fixed-dyn.mp4

I have heard that changing the first ~180 hex values can have an effect. What do you think?

I am really grateful for your time,

anthwlock commented 4 years ago

The untrunc -sh said command not found.

Because the untrunc executable is in the container, please try docker run -v ~/Downloads/Videos/:/mnt untrunc -sh /mnt/C0002.mp4

I have heard that changing the first ~180 hex values can have an effect. What do you think?

If you are really lucky, you can repair a file by just copy-pasting the moov atom. If the moov atom is located in the first ~180 bytes, this could work. May I ask where you have read this?

broncobenshea commented 4 years ago

So, I copied the first 180 bytes from a non corrupted file to the corrupted file. ran your program and IT WORKED! The video is a little jittery I assume some frames are missing but otherwise definitely something we can cut to. Thank you so much! sending you a case of beer!

anthwlock commented 4 years ago

Happy to hear it worked. I looked at the log again, and (believe to) understand the issue now: The mdat atom has constant offset, but the needed mp4-structure does not exist in the corrupt file. This is very similar to the XAVC files I once received. https://github.com/anthwlock/untrunc/blob/d314f8f09332b65456ac951b28d68f6b27b46122/src/mp4.cpp#L893-L895 It would probably be a good idea to make this check more generic. Perhaps always if mdat comes before moov.

If you don't mind, it would nice to have your two files (first ~10mb are enough) so I could test this. You can use: docker run -v ~/Downloads/Videos/:/mnt untrunc -sh /mnt/C0003.mp4 10 docker run -v ~/Downloads/Videos/:/mnt untrunc -sh /mnt/C0002.mp4 10 to extract first 10mb, and then upload them to wetransfer.com

Again, happy to hear that it worked, and if you don't want me to send the files that's ok too. (And thank you especially for your kind donation!)

broncobenshea commented 4 years ago

I'll work on sending the files

any ideas on how to smooth the frames?

what email should I send them to?

anthwlock commented 4 years ago

Have you processed C0002.mp4 any further? Or maybe they are from two different cameras? Because while C0003.mp4 looks like a 'iso mp4' file, C0002.mp4 seems to be in XAVC format. This is directly visible by comparing the ftyp (filetype) atom.

I have trouble to recovery anything at all, even after replacing the first 180 bytes. You didn't use the -s parameter, did you? Since the video frames are unexpectedly big (very high resolution), a sample bigger than 10mb might be useful. I'd say 200mb should be enough. docker run -v ~/Downloads/Videos/:/mnt untrunc -sh /mnt/C0003.mp4 200 docker run -v ~/Downloads/Videos/:/mnt untrunc -sh /mnt/C0002.mp4 200 to extract first 200mb, and then upload them to wetransfer.com (just like before).

Btw, can you further describe what you mean with 'smooth frames'? Visual artifacts or just missing frames?

anthwlock commented 4 years ago

got no files -> closing