search

holepunchto / hyperdrive

Hyperdrive is a secure, real time distributed file system
Apache License 2.0
1.86k stars 135 forks source link

Missing NTFS/ReFS sparse support #267

Closed da2x closed 1 year ago

da2x commented 4 years ago

NTFS requires files to be created with the FILE_ATTRIBUTE_SPARSE_FILE attribute to treat them as sparse. This is different from other modern OSes where you don’t need to do anything special.

I tested with fsutil.exe sparse queryflag hyperdrive_file_name and various calls to check file sizes, and found that Hyperdrives aren’t marked as sparse files on NTFS.

Because of this, Hyperdrives downloaded with Dat or Beaker take up the full size of the archive on disk even though it contains huge sections of zero-bytes.

mafintosh commented 4 years ago

Excellent find! I’ll try to fix this in the random-access-file module

da2x commented 4 years ago

Sorry about reporting against the wrong repo. Should I close this and reopen an issue there?

mafintosh commented 4 years ago

If you don’t mind yes. We can keep this one open to track it here also still

da2x commented 4 years ago

https://github.com/random-access-storage/random-access-file/issues/23