search

Yutaka-Sawada / MultiPar

Parchive tool
977 stars 43 forks source link

recovery of big file failed #129

Closed DNFrozen1 closed 4 months ago

DNFrozen1 commented 5 months ago

Multipar version 1.3.3.2

I took a big 522GB file (to be exact 560.583.868.494 Bytes) and split it into 10000 blocks and created par2 files with 300% redundancy divided into 120 files so 60 hours later (HDD) i had 120 files each containing 250 blocks (total 30000 blocks).

but then i decided to test the recovery process so i moved the original file to a different folder and tried to reconstruct it entirely using the par2 files. the recovery process took about a day and ended with "failed to repair"

Yutaka-Sawada commented 5 months ago

Thank you for bug report. Because I cannot test such big file myself, I'm not sure what is a problem. When I tested smaller files with 10000 source blocks, there was no problem. So, big file size might cause something unknown error.

I made debug version to see internal behavior. Though the latest version is v1.3.3.3, the recoverying mechanism is same as old v1.3.3.2. It doesn't solve any problem, but just shows what is going. If you have time to test, please try. I put the debug package (par2j_debug_2024-06-05.zip) in "MultiPar_sample" folder on OneDrive.

When you are busy and cannot test, you may be good to avoid big files on MultiPar. If a big source file is an archive, you would better keep smaller archives. Normally, protecting small file is faster and easier than big file. So, treating 10 small files independently may be better than 1 big file. But, it depends on your usage. If you don't need GUI, you may try another PAR2 tool; par2cmdline-turbo. Because it runs in different mechanism from par2j, it may work for your case. It's very fast on recent PC as same as my par2j.

DNFrozen1 commented 5 months ago

Thanks for your help So the new program just finished the recovery process and this time it did not fail. the file was restored without issues. I'm now trying to reproduce the bug with the old exe file (will take a day again). I'll let you know the results

but is seems like the bug is fixed or never existed in the first place. I have no idea why it failed last time

DNFrozen1 commented 4 months ago

This time the old exe file sucessfully restored the file. I don't know what circumstances caused it to fail last time.

I'm sorry for wasting your time

Yutaka-Sawada commented 4 months ago

This time the old exe file sucessfully restored the file.

Oh, I see. Because the recovery consumes long time, another tool or Windows OS might prevent a file access. Someone reported similar timing issue at big file handling ago. While Anti-Virus software is scanning a very big file, MultiPar cannot rename, move, or write the locked file. It would cause failed repair or verification. For small files, Anti-Virus software finishes quickly. When file access takes long time for big files, exclusive file handling happens to be a problem.

But, I'm not sure what was wrong in your case realy, too. If you see the same problem again, please report later. Saving log in normal usage may be good to see what happend. If this was the timing issue, I may increase waiting time for locked files.

I'm sorry for wasting your time

There is no problem for me. Thank you for testing again and confirm the behavior.