search

gyroflow / mp4-merge

A tool and library to losslessly join multiple .mp4 files shot with same camera and settings
Apache License 2.0
65 stars 5 forks source link

Consider buffered I/O #5

Closed racer-coder closed 1 month ago

racer-coder commented 1 month ago

mp4-merge seems really slow on Windows 11 (also even worse on Linux under WSL2 on Windows 11, but I assume that's a WSL2 issue). Windows 11: ~70 seconds to create a 4.3GB file Ubuntu 22.04 under WSL2 on Windows 11: 300+ seconds for the same 4.3GB file.

I believe this may be due to not using buffered I/O, but I don't know Rust enough to tell. I did find this link in case it helps:

https://nnethercote.github.io/perf-book/io.html

In contrast, using LossLessCut to select a 3GB section from that 4.3GB file took less than 10 seconds.

AdrianEddy commented 1 month ago

Try latest master (compile from source) and let me know if it's faster for you. Make sure to build in release mode cargo build --release

racer-coder commented 1 month ago

That is a substantial improvement for Ubuntu 22.04 under WSL2 on Windows 11 (note the source files are on NTFS while the target is stored on the "ext4" Linux filesystem):

master:

scott@TRACKTOP2:~/src/mp4-merge$ time ./target/release/mp4_merge /mnt/c/Users/Sco tt\ Smith/mp4mergetest/GH0* --out merged-fast.mp4 Merging file "/mnt/c/Users/Scott Smith/mp4mergetest/GH012649.MP4" Merging file "/mnt/c/Users/Scott Smith/mp4mergetest/GH022649.MP4" Merging file "/mnt/c/Users/Scott Smith/mp4mergetest/GH032649.MP4" Output file "merged-fast.mp4" Done in 142.018s

real 2m22.031s user 0m0.547s sys 0m18.990s

v0.1.9:

scott@TRACKTOP2:~/src/mp4-merge$ time ./target/release/mp4_merge /mnt/c/Users/Scott\ Smith/mp4mergetest/GH0* --out merged-slow.mp4 Merging file "/mnt/c/Users/Scott Smith/mp4mergetest/GH012649.MP4" Merging file "/mnt/c/Users/Scott Smith/mp4mergetest/GH022649.MP4" Merging file "/mnt/c/Users/Scott Smith/mp4mergetest/GH032649.MP4" Output file "merged-slow.mp4" Done in 506.263s

real 8m26.273s user 0m3.407s sys 0m51.220s

Output file comparison:

scott@TRACKTOP2:~/src/mp4-merge$ cmp merged-fast.mp4 merged-slow.mp4 ; echo $? 0

I'm not sure how accurate the results of 'time' are here, though I can attest 'real' is pretty accurate. Maybe the discrepancy between user+sys and real is the glue from WSL2 to Windows.

Anyways, thank you, much appreciated!

AdrianEddy commented 1 month ago

Ok v0.1.10 is released