search

TheDiscordian / ipfs-sync

A simple daemon which will watch files on your filesystem, mirror them to MFS, automatically update related pins, and update related IPNS keys.
BSD 3-Clause "New" or "Revised" License
94 stars 14 forks source link

Bulk add directories for initial sync #31

Open TheDiscordian opened 3 years ago

TheDiscordian commented 3 years ago

As directory size grows, calls to MFS like files/cp, files/mkdir, and files/rm get slower and slower. For initial sync at least, adding directories, we could add all the files at once, with the entire directory tree. Then a single files/cp could copy the tree into MFS, solving this problem for initial sync.

Could still be a bit slow for anything other than initial-sync, but I think initial sync is where this will impact the most.

TheDiscordian commented 3 years ago

files/cp doesn't seem too slow. Will need more testing, but with #28 and #29 tackled, this issue doesn't impact nearly as much.