search

yadayada / acd_cli

An unmaintained command line interface and FUSE filesystem for Amazon (Cloud) Drive
Other
1.35k stars 165 forks source link

acd_cli upload: Slow with lots of files #483

Closed hunger closed 8 years ago

hunger commented 8 years ago

I am using "acd_cli upload --overwrite" to upload a folder with about 100k tiny files in various subfolders.

This use-case is largely dominated by the time it takes to find out which files need uploading. A test with just 10k files takes about 40min, even if no file needs to be uploaded. The result is similar when I leave out the "--overwrite" parameter.

Are there some tricks to speed up the process? I tried using "-x 2" already, but that does not seem to effect this work load. In fact it seems to slow down the upload a bit.

yadayada commented 8 years ago

I have merged the PR #442 that addresses this issue.