search

xyou365 / AutoRclone

AutoRclone: rclone copy/move/sync (automatically) with thousands of service accounts
https://www.gfan.loan/?p=235
1.36k stars 517 forks source link

Slow copy while checking files #53

Open vinayorama opened 4 years ago

vinayorama commented 4 years ago

hi.. i have Gdrive with 500TB data.. i used Autoclone to copy that into another drive.

now i have added additional 50TB data into my original drive. but when using autoclone to copy into another drive.. its taking forever to check existing data.. how to increase performance of check process. .do we need to add additional flag while running it ?

plz help

Priyansh2 commented 3 years ago

@vinayorama I'm assuming you have already figured this out and in case you not, AutoRclone uses Rclone and so you have to add extra flags and commands inside the rclone_sa_magic.py file. While copying stuff you can add --checksum or some other global flags like --size-only, --no-update-modtime which are well documented in the official Rclone doc. These flags will help you to prevent duplicate copies of the already existing files in your destination. If you just want to check your destination whether it contains all files and folder present in your source, you can use --one-way flag with the check command: rclone check <src_path> <destination_path>. Additional you can use --checksum or --size-only (global flags). For more info regarding this, you can view Rclone doc.