anthwlock
commented
4 years ago @galaxy001 If you send me the files (healthy+broken), I will have a look.
Closed galaxy001 closed 4 years ago
anthwlock
commented
4 years ago @galaxy001 If you send me the files (healthy+broken), I will have a look.
galaxy001
commented
4 years ago I got a pair. Please use patch.sh to generate the bad file with patch.
https://send.firefox.com/download/1c9274e2afb99b95/#50aOTFnppwSUNX2zdUzx4w, this link allows downloads for 3 times within 7days.
willsc123
commented
4 years ago Hi, I have the same problem with some drone clips I recovered from a formatted drive. ` Info: version 'ebeb1a1-dirty' using ffmpeg '3.4.6-0ubuntu0.18.04.1' Info: reading /mnt/DJI_0378.MOV Info: parsing healthy moov atom ...
Info: unknown track 'hvc1' found -> fallback to dynamic stats Error: bad track: 'hvc1' ` I'm running untrunc from a Docker container on CentOS 7. I would like to know how to recover the files myself since there are a lot of them. I've been able to partially recover some of them but the length is shorter and some frames are skipping. Thanks
anthwlock
commented
4 years ago @galaxy001 I wonder why the healthy and broken have the same content.. Have you just manually removed the moov atom from the healthy one? Please try the new master, it works for me.
@willsc123 What do you mean with 'the length is shorter'? The filesize? By how much? If you still have problems after thew new commits, please send me your files (don't forget a reference file).
willsc123
commented
4 years ago Hi Anthon, thanks for the quick respons, Yes, using other tools I've been able to recover the file but its size is 60% of the original and it skips frames every second or so. I've tried to build untrunc again with the latest commit (using docker because on CentOS I can't figure out how to build it from source code, I get errors with libavcodec missing). Now it fails finding the correct codec and terminates prematurely. I'll attach the output of the command. Please let me know how to send you the files. Thanks
$ sudo docker run -v /home/will/Descargas/dronCarolinDani/:/mnt untrunc -vv /mnt/DJI_0378.MOV /mnt/000205.mov
[NULL @ 0x55cd9af4d080] Opening '/mnt/DJI_0378.MOV' for reading
[file @ 0x55cd9af4d920] Setting default whitelist 'file,crypto'
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55cd9af4d080] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55cd9af4d080] ISO: File Type Major Brand: qt
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55cd9af4d080] Unknown dref type 0x206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55cd9af4d080] Before avformat_find_stream_info() pos: 1459921127 bytes read:54843 seeks:1 nb_streams:1
[hevc @ 0x55cd9af4e540] nal_unit_type: 32(VPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0x55cd9af4e540] Decoding VPS
[hevc @ 0x55cd9af4e540] Main 10 profile bitstream
[hevc @ 0x55cd9af4e540] nal_unit_type: 33(SPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0x55cd9af4e540] Decoding SPS
[hevc @ 0x55cd9af4e540] Main 10 profile bitstream
[hevc @ 0x55cd9af4e540] Decoding VUI
[hevc @ 0x55cd9af4e540] nal_unit_type: 34(PPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0x55cd9af4e540] Decoding PPS
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55cd9af4d080] All info found
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55cd9af4d080] After avformat_find_stream_info() pos: 1046442 bytes read:1101249 seeks:2 frames:1
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/mnt/DJI_0378.MOV':
Metadata:
major_brand : qt
minor_version : 512
compatible_brands: qt
creation_time : 2019-09-12T18:35:49.000000Z
encoder : Lavf56.15.102
Duration: 00:01:56.91, start: 0.000000, bitrate: 99901 kb/s
Stream #0:0(eng), 1, 1/24000: Video: hevc (Main 10), 1 reference frame (hvc1 / 0x31637668), yuv420p10le(tv), 3840x2160, 0/1, 99900 kb/s, 23.98 fps, 23.98 tbr, 24k tbn, 23.98 tbc (default)
Metadata:
creation_time : 2019-09-12T18:35:49.000000Z
handler_name : DataHandler
Info: version '2b01278-dirty' using ffmpeg '3.4.6-0ubuntu0.18.04.1'
Info: reading /mnt/DJI_0378.MOV
Info: parsing healthy moov atom ...
requests: 4 at offset : 0
requests: 4 at offset : 4
start_ = 0
length_ = 20
name_ = ftyp
requests: 12 at offset : 8
requests: 4 at offset : 20
requests: 4 at offset : 24
start_ = 20
length_ = 8
name_ = wide
requests: 0 at offset : 28
requests: 4 at offset : 28
requests: 4 at offset : 32
start_ = 28
length_ = 1459899024
name_ = mdat
requests: 4 at offset : 0
requests: 4 at offset : 4
start_ = 1459899052
length_ = 22075
name_ = moov
requests: 4 at offset : 8
requests: 4 at offset : 12
start_ = 1459899060
length_ = 108
name_ = mvhd
requests: 100 at offset : 16
requests: 4 at offset : 116
requests: 4 at offset : 120
start_ = 1459899168
length_ = 21914
name_ = trak
requests: 4 at offset : 124
requests: 4 at offset : 128
start_ = 1459899176
length_ = 92
name_ = tkhd
requests: 84 at offset : 132
requests: 4 at offset : 216
requests: 4 at offset : 220
start_ = 1459899268
length_ = 21814
name_ = mdia
requests: 4 at offset : 224
requests: 4 at offset : 228
start_ = 1459899276
length_ = 32
name_ = mdhd
requests: 24 at offset : 232
requests: 4 at offset : 256
requests: 4 at offset : 260
start_ = 1459899308
length_ = 45
name_ = hdlr
requests: 37 at offset : 264
requests: 4 at offset : 301
requests: 4 at offset : 305
start_ = 1459899353
length_ = 21729
name_ = minf
requests: 4 at offset : 309
requests: 4 at offset : 313
start_ = 1459899361
length_ = 20
name_ = vmhd
requests: 12 at offset : 317
requests: 4 at offset : 329
requests: 4 at offset : 333
start_ = 1459899381
length_ = 44
name_ = hdlr
requests: 36 at offset : 337
requests: 4 at offset : 373
requests: 4 at offset : 377
start_ = 1459899425
length_ = 36
name_ = dinf
requests: 4 at offset : 381
requests: 4 at offset : 385
start_ = 1459899433
length_ = 28
name_ = dref
requests: 20 at offset : 389
requests: 4 at offset : 409
requests: 4 at offset : 413
start_ = 1459899461
length_ = 21621
name_ = stbl
requests: 4 at offset : 417
requests: 4 at offset : 421
start_ = 1459899469
length_ = 233
name_ = stsd
requests: 225 at offset : 425
requests: 4 at offset : 650
requests: 4 at offset : 654
start_ = 1459899702
length_ = 24
name_ = stts
requests: 16 at offset : 658
requests: 4 at offset : 674
requests: 4 at offset : 678
start_ = 1459899726
length_ = 392
name_ = stss
requests: 384 at offset : 682
requests: 4 at offset : 1066
requests: 4 at offset : 1070
start_ = 1459900118
length_ = 4432
name_ = stsc
requests: 4424 at offset : 1074
requests: 4 at offset : 5498
requests: 4 at offset : 5502
start_ = 1459904550
length_ = 11232
name_ = stsz
requests: 11224 at offset : 5506
requests: 4 at offset : 16730
requests: 4 at offset : 16734
start_ = 1459915782
length_ = 5300
name_ = stco
requests: 5292 at offset : 16738
requests: 4 at offset : 22030
requests: 4 at offset : 22034
start_ = 1459921082
length_ = 45
name_ = udta
requests: 37 at offset : 22038
ftyp_ = 'qt '
assuming constant duration of 1001 for 'hvc1' (x2803)
calling findMdat on truncated file..
requests: 4 at offset : 0
requests: 4 at offset : 4
start_ = 0
length_ = 20
name_ = ftyp
requests: 4 at offset : 20
requests: 4 at offset : 24
start_ = 20
length_ = 8
name_ = wide
requests: 4 at offset : 28
requests: 4 at offset : 32
start_ = 28
length_ = 301569337
name_ = mdat
Info: reading mdat from truncated file ...
(reading element from mdat)
Offset: 0 / 36 : 00000023 40010c01
Track codec: hvc1
---
Length: 39
Nal type: 32
nuh_layer_id: 0
nuh_temporal_id_plus1: 1
Partial hvc1-length: 39
---
Length: 47
Nal type: 33
nuh_layer_id: 0
nuh_temporal_id_plus1: 1
Partial hvc1-length: 86
---
Length: 11
Nal type: 34
nuh_layer_id: 0
nuh_temporal_id_plus1: 1
Warning: Too short!
failed parsing h256 nal-header
part-length: 86
1th sample in 1th hvc1-chunk
(reading element from mdat)
Offset: 86 / 122 : 00000007 4401c1ad
Track codec: hvc1
---
Length: 11
Nal type: 34
nuh_layer_id: 0
nuh_temporal_id_plus1: 1
Warning: Too short!
failed parsing h256 nal-header
part-length: 0
Invalid length: part-length is 0
Error: unable to find correct codec -> premature end (~2.852e-05%)
try '-s' to skip unknown sequences
mdat->file_end: 122
Info: Found 1 packets ( hvc1: 1 hvc1-keyframes: 0 )
Info: Duration of hvc1: 41ms (41 ms)
Info: saving /mnt/000205.mov_fixed.MOV@willsc123 just uplaod them somewhere (e.g. https://wetransfer.com/) and send me the link (email)
anthwlock
commented
4 years ago @willsc123
After ce14b06, I was able to recover your file using -s.
Hope it also works for the rest of your files
willsc123
commented
4 years ago Yes! It works! I've tried recompiling on CentOS using Docker but the command didn't output nothing but
Info: version 'ce14b06' using ffmpeg '3.3.4'
Info: reading .\DJI_0370.MOV
Info: parsing healthy moov atom ...And it jumps back to console prompt. Maybe it's something with a bad build (I haven't tried on Ubuntu). Then I've tried using the executable for Windows and it works flawlessly using -s. I've managed to recover pretty much all my drone clips apart from some minor artifacts and glitchy beginnings (missing keyframes) which I don't mind of.
You save my life! Awesome tool! Thanks!
galaxy001
commented
4 years ago @anthwlock It works. I am using FlashAir WiFI SD card with an microSD to SD adapter. So I can dump file while the card is inside the camera.
And I got another situation. The GoPro camera will attempt to fix unfinished file itself on next boot.
However, sometime it will write a moov with much shorter time table to the file and thus make a 5-min video turns into a few-seconds one. Could you add an option to ignore the moov atom and regenerate a correct one based on the file content ?
$ ls -lL bad.mp4
-rw-r--r-- 1 galaxy galaxy 4011748847 Mar 17 04:46 bad.mp4
$ mediainfo bad.mp4
General
Complete name : bad.mp4
Format : MPEG-4
Format profile : Base Media / Version 1
Codec ID : mp41 (mp41)
File size : 3.74 GiB
Duration : 28 s 0 ms
Overall bit rate : 1 146 Mb/s
Encoded date : UTC 2020-03-10 21:31:28
Tagged date : UTC 2020-03-10 21:31:28
xyz : +22.5871+114.2602/
IsTruncated : Yes
Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main@L6@Main
Codec ID : hvc1
Codec ID/Info : High Efficiency Video Coding
Duration : 320 ms
Bit rate : 104 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 25.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Bits/(Pixel*Frame) : 0.500
Stream size : 3.95 MiB (0%)
Title : GoPro H.265
Language : English
Encoded date : UTC 2020-03-10 21:31:28
Tagged date : UTC 2020-03-10 21:31:28
Color range : Full
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
Codec configuration box : hvcC
Other #1
ID : 2
Type : Time code
Format : QuickTime TC
Duration : 320 ms
Bit rate mode : Constant
Frame rate : 25.000 FPS
Time code of first frame : 21:42:09:02
Time code, striped : Yes
Title : GoPro TCD
Language : English
Encoded date : UTC 2020-03-10 21:31:28
Tagged date : UTC 2020-03-10 21:31:28
Other #2
Type : meta
Duration : 28 s 0 ms
Bit rate mode : Constant
Other #3
Type : meta
Bit rate mode : Variable
mdhd_Duration : 320
$ untrunc -f good.mpv bad.mp4
0: ftyp (20)
20: mdat (4011815564)
4011815584: moov (285976)
4011815592: mvhd (108)
4011815700: udta (26421)
4011841989: free (132)
4011842121: iods (24)
4011842145: trak (65941)
4011842153: tkhd (92)
4011842245: tref (20)
4011842253: tmcd (12)
4011842265: edts (36)
4011842273: elst (28)
4011842301: mdia (65785)
4011842309: mdhd (32)
4011842341: hdlr (44)
4011842385: minf (65701)
4011842393: vmhd (20)
4011842413: dinf (36)
4011842421: dref (28)
4011842457: stsd (229)
4011842686: stts (24)
4011842710: stsc (28)
4011842738: stsz (32052)
4011874790: stco (32048)
4011906838: stss (1248)
4011908086: trak (464)
4011908094: tkhd (92)
4011908186: mdia (364)
4011908194: mdhd (32)
4011908226: hdlr (44)
4011908238: tmcd (1835560050) <-- out of range
4011908270: minf (280)
4011908278: gmhd (82)
4011908360: dinf (36)
4011908368: dref (28)
4011908396: stbl (154)
4011908404: stsd (50)
4011908420: tmcd (34)
4011908454: stts (24)
4011908478: stsc (28)
4011908506: stsz (24)
4011908530: stco (20)
4011908550: trak (64468)
4011908558: tkhd (92)
4011908650: mdia (64368)
4011908658: mdhd (32)
4011908690: hdlr (44)
4011908702: meta (1835560050) <-- out of range
4011908734: minf (64284)
4011908742: gmhd (44)
4011908786: dinf (36)
4011908794: dref (28)
4011908822: stbl (64196)
4011908830: stsd (36)
4011908866: stts (24)
4011908890: stsc (28)
4011908918: stsz (32052)
4011940970: stco (32048)
4011973018: trak (128542)
4011973026: tkhd (92)
4011973118: mdia (128442)
4011973126: mdhd (32)
4011973158: hdlr (44)
4011973170: meta (1835560050) <-- out of range
4011973202: minf (128358)
4011973210: gmhd (40)
4011973250: dinf (36)
4011973258: dref (28)
4011973286: stbl (128274)
4011973294: stsd (34)
4011973328: stts (24)
4011973352: stsc (28)
4011973380: stsz (64092)
4012037472: stco (64088)untrunc -s works, but the beginning of the fixed video contains some visual artifact when played.
$ untrunc -s good.mpv bad.mp4
WARNING: Because of internal decoder changes, using ffmpeg 'n4.2.2' with '-s' can be slow!
You are advised to compile untrunc against ffmpeg 3.3!
See the README.md on how to do that.
Info: version 'ce14b06' using ffmpeg 'n4.2.2'
Info: reading good.mpv
Info: parsing healthy moov atom ...
Info: special track found (tmcd, 'GoPro TCD')
Info: special track found (meta, 'GoPro MET')
Info: special track found (meta, 'GoPro SOS')
Info: reading mdat from truncated file ...
Info: Found 31884 packets ( gpmd: 7968 fdsc: 15944 hvc1: 7971 hvc1-keyframes: 307 tmcd: 1 )
Info: Duration of gpmd: 5min 18s 720ms (318720 ms)
Info: Duration of hvc1: 5min 18s 840ms (318840 ms)
Warning: Unknown sequences: 10
Warning: Bytes NOT matched: 468KiB (0.01195%)
Info: saving bad.mp4_fixed-s1.mpv
2 warnings were hidden!
$ ls -lL bad.mp4*
-rw-r--r-- 1 galaxy galaxy 4011748847 Mar 17 04:46 bad.mp4
-rw-r--r-- 1 galaxy galaxy 4011268285 Mar 17 05:01 bad.mp4_fixed-s1.mpv
$ mpv bad.mp4_fixed-s1.mpv
(+) Video --vid=1 (*) (hevc 3840x2160 25.000fps)
[ffmpeg/video] hevc: Invalid NAL unit 0, skipping.
[ffmpeg/video] hevc: Invalid NAL unit 0, skipping.
[ffmpeg/video] hevc: Invalid NAL unit 0, skipping.
[ffmpeg/video] hevc: Invalid NAL unit 0, skipping.
... ...@galaxy001
Could you add an option to ignore the
moovatom and regenerate a correct one based on the file content ?
The second file is getting repaired, no matter if it has moov or not.
The first file needs to have a moov atom to guess general structure of the second file.
In your case, you could even do untrunc -n bad.mp4 bad.mp4.
untrunc -sworks, but the beginning of the fixed video contains some visual artifact when played.
Again, please send me the first ~70mb of both files. Is it the case, that the video recovered by GoPro itself does contain the beginning without these artifacts? If so, please make sure to include that file.
galaxy001
commented
4 years ago Mail sent.
$ AtomicParsley bad80m.mp4 -T 1
Atom ftyp @ 0 of size: 20, ends @ 20
Atom mdat @ 20 of size: 4153266, ends @ 4153286
Atom moov @ 4153286 of size: 28732, ends @ 4182018It seems GoPro have overwritten a moov inside the file.
anthwlock
commented
4 years ago @galaxy001
The moov atom written by the GoPro recovery program has overwritten the actual data (I call this bad recovery).
So your artifacts stem from missing data, nothing can be done. Or do you have the same file before GoPro tried to recover it?
To be clear, the artifacts reported by @willsc123 have a different reason, likely wrong (too big) size detection of the hvc1 samples.
anthwlock
commented
4 years ago Ok, it looks like somehow the first ~1MB (of bad.mov from @willsc123) are "disturbed".
They contain multiple sequences of just zero bytes, found nowhere else, neither after the first ~1MB, nor in the reference file.
The zero sequences are:
141788,141826 -> 38
669469,669698 -> 229
1117382,1117698 -> 316I fear that not much can be done here.. luckily this occurs only at the the beginning. It would be interesting to know why this happens, and only in the first ~1mb.
Would you add support for GoPro mp4 files ?
It is a time-lapse video, so no audio presents.
The mp4 file will have no
moovwhen there is a sudden power lost.