Hashing now works on NTFS! Dumps can be completed in only 34 minutes on NTFS, even with the additional computations required for hashing.
The resulting WUD files are bit-for-bit identical to ones dumped on FAT32, so it seems safe to say that NTFS writing is still as accurate as it was before.
I have attached a compiled binary:
wudump.zip
SHA-256:
6db85b143708aacd4d2bf06156775c0cc90a4f5af8df466d98d3e21d04e38666 wudump.elf
Note:
There are a lot of problems compiling wudump on the newest devkitPro, so I had to compile it with devkitPPC r29 and devkitARM r46 instead. I will try to make it work with the newest version of devkitPro in the future. If I am successful, I will make another PR to bring things up to date. Wish me luck!
Background:
@Maschell's pull request mentioned that hashing was disabled due to a soft lock issue.
The reason the soft lock occurred was the use of while(!OSIsThreadSuspended(thread));. On FAT32, this ensures the hashing thread finishes before the next WUD part file is written. For FAT32, that is perfectly fine. That is not the case for NTFS, however!
On NTFS, the WUD file is not split. The hashing thread runs continuously until the full 25GB is copied. Since the code mentioned above pauses the copy operation, the hashing thread starts waiting for newly copied data. That data is never received because the current copy operation is paused. The end result is a soft lock.
@Maschell was right to disable that particular line of code on NTFS. The trick was to ONLY leave this line disabled. All other code related to hashing can be safely enabled on NTFS. (Since the part hashes would end up being the same as the global hash, I still left part hashing disabled.)
In spite of hashing being enabled again, NTFS performance is still much better than FAT32 performance. I originally suspected (like @Maschell) that hashing was the reason why wudump's FAT32 performance was worse, so this surprised me a little. I don't know much about filesystems, but I'm guessing that the libfat library simply has performance problems.
Fixes #17
Hashing now works on NTFS! Dumps can be completed in only 34 minutes on NTFS, even with the additional computations required for hashing.
The resulting WUD files are bit-for-bit identical to ones dumped on FAT32, so it seems safe to say that NTFS writing is still as accurate as it was before.
I have attached a compiled binary: wudump.zip SHA-256:
6db85b143708aacd4d2bf06156775c0cc90a4f5af8df466d98d3e21d04e38666 wudump.elfNote:
There are a lot of problems compiling wudump on the newest devkitPro, so I had to compile it with devkitPPC r29 and devkitARM r46 instead. I will try to make it work with the newest version of devkitPro in the future. If I am successful, I will make another PR to bring things up to date. Wish me luck!
Background:
@Maschell's pull request mentioned that hashing was disabled due to a soft lock issue.
The reason the soft lock occurred was the use of
while(!OSIsThreadSuspended(thread));. On FAT32, this ensures the hashing thread finishes before the next WUD part file is written. For FAT32, that is perfectly fine. That is not the case for NTFS, however!On NTFS, the WUD file is not split. The hashing thread runs continuously until the full 25GB is copied. Since the code mentioned above pauses the copy operation, the hashing thread starts waiting for newly copied data. That data is never received because the current copy operation is paused. The end result is a soft lock.
@Maschell was right to disable that particular line of code on NTFS. The trick was to ONLY leave this line disabled. All other code related to hashing can be safely enabled on NTFS. (Since the part hashes would end up being the same as the global hash, I still left part hashing disabled.)
In spite of hashing being enabled again, NTFS performance is still much better than FAT32 performance. I originally suspected (like @Maschell) that hashing was the reason why wudump's FAT32 performance was worse, so this surprised me a little. I don't know much about filesystems, but I'm guessing that the
libfatlibrary simply has performance problems.