search

anthwlock / untrunc

Restore a truncated mp4/mov. Improved version of ponchio/untrunc
GNU General Public License v2.0
1.9k stars 182 forks source link

The README seriously undersells just how much better this fork is. Literally over 800x improvement for me. #70

Closed happysmash27 closed 2 years ago

happysmash27 commented 3 years ago

Your version of untrunc, runs in just a couple of minutes, the same videos that I had been running the original untrunc on for over 17 hours with it still not finishing. Literally, 2 minutes vs maybe 12 hours (I didn't see exactly when it finished) for the first video, maybe 3 vs over 17 for the second. It's like, literally over 500x faster than the original, not merely 10!

So, firstly, thank you! This is saving me hours of time!

And secondly, so that this is technically an issue and not just a thank you, perhaps you should mention in the README, that this can actually be hundreds, maybe even thousands of times faster than the original, not merely 10x, and that's not even with multithreading!

Compilation is way faster too. My, I wish I knew about this approximately 18 hours ago! I've been waiting for more than 17 hours with all these files when I could have untruncated them in just a couple minutes!

1354041 packets. Your fork? 2 minutes 25 seconds. The original? Still not done after 17 hours!!!

Actually, wait, one of the old ones is finally finishing after 17 hours 17 minutes!

1341206 packets.

Let's try your untrunc, shall we?

1 minute, 14 seconds. On what used to take over 17 hours. That is almost an 1000x improvement! And half that time is spent on hard disk I/O instead of calculation!

Your README seriously undersells it. There is a nice progress indicator too instead of endless logs of basically the same thing.

Oh, and I no longer get I/O errors in MPV too.

Thank you so much.

Maybe add it to the README?

anthwlock commented 3 years ago

Glad to hear you like it.

perhaps you should mention in the README, that this can actually be hundreds, maybe even thousands of times faster than the original, not merely 10x

I have doubts..

  1. The improvement factor is file dependent
  2. For proper comparison you need to consider the pagecache.. Not sure if you have done that.

In my tests, this fork is around 5 to 20 times faster.

rubo77 commented 3 years ago

Is this still the case? the orignal has a new branch called "logging" where there is a lot of improvements by now

anthwlock commented 3 years ago

Is this still the case? the orignal has a new branch called "logging" where there is a lot of improvements by now

Seems so. I wrote a little benchmark, and tested it with 4 different files. I expected both versions would be able to recover all of them, but apparently I was wrong (see results below). All video files lied on a slow HDD. Also keep in mind that many files are not recoverable with the original untrunc project at all (including untrunc-logging). That's because I received many additional samples for which untrunc failed, and then adapted the code so they are supported (by this fork). That said, it is always possible that the original project gets extended support for files this fork currently does not support. But I usually try to port such changes quickly, given I have the required time.

Here are the results:

anth nocache: [mp4a, avc1] 38.1MiB took  0.86s ~> pkgs=1320 audio=19.09s video=16.43s
logg nocache: [mp4a, avc1] 38.1MiB took  1.85s ~> pkgs=1320 audio=19.09s video=13.72s
anth   cache: [mp4a, avc1] 38.1MiB took  0.10s ~> pkgs=1320 audio=19.09s video=16.43s
logg   cache: [mp4a, avc1] 38.1MiB took  1.00s ~> pkgs=1320 audio=19.09s video=13.72s

anth nocache: [mp4a, avc1] 176.7MiB took  1.98s ~> pkgs=8246 audio=0 video=5m 29.84s
logg nocache: [mp4a, avc1] 176.7MiB took  6.90s ~> pkgs=8246 audio=0 video=0
anth   cache: [mp4a, avc1] 176.7MiB took  0.23s ~> pkgs=8246 audio=0 video=5m 29.84s
logg   cache: [mp4a, avc1] 176.7MiB took  3.67s ~> pkgs=8246 audio=0 video=0

anth nocache: [mp4a, avc1] 381.5MiB took  6.97s ~> pkgs=23410 audio=5m 04.51s video=5m 04.84s
logg nocache: [mp4a, avc1] 381.5MiB took 17.95s ~> pkgs=23410 audio=3m 31.44s video=3m 31.44s
anth   cache: [mp4a, avc1] 381.5MiB took  0.90s ~> pkgs=23410 audio=5m 04.51s video=5m 04.84s
logg   cache: [mp4a, avc1] 381.5MiB took  5.21s ~> pkgs=23410 audio=3m 31.44s video=3m 31.44s

anth nocache: [avc1] 190.7MiB took  3.78s ~> pkgs=15143  video=22m 50.63s
logg nocache: [avc1] 190.7MiB took  2.30s ~> pkgs=0  video=-
anth   cache: [avc1] 190.7MiB took  2.51s ~> pkgs=15143  video=22m 50.63s
logg   cache: [avc1] 190.7MiB took  0.14s ~> pkgs=0  video=-

Tested with anth 8024f0cfbbc94c70a30004726fbfa6b23baa8b63 and logg https://github.com/ponchio/untrunc/commit/be5d007632c437bff4291fd92351fc5ca428f095