search

opencadc / vostools

VOSpace command line and FUSE clients
https://www.canfar.net/en/docs/storage
12 stars 18 forks source link

vmv slow when moving large-ish data volumes (~TB) #205

Open drtobybrown opened 2 years ago

drtobybrown commented 2 years ago

vmv vos:path/data vos:newpath/data takes several minutes for 0.5 TBs

exalate-issue-sync[bot] commented 2 years ago

Adrian Damian commented: How many files are in there? vmv execution time does not depend on the size but it's influenced by the number of files.

Thanks, Adrian

drtobybrown commented 2 years ago

Not sure exactly but likely in the 1000s

On Fri, May 20, 2022 at 08:41 exalate-issue-sync[bot] < @.***> wrote:

Adrian Damian commented: How many files are in there? vmv execution time does not depend on the size but it's influenced by the number of files.

Thanks, Adrian

— Reply to this email directly, view it on GitHub https://github.com/opencadc/vostools/issues/205#issuecomment-1133049876, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQK7ALJBVV4BLPNXYP7QWDVK6XCFANCNFSM5WDDB6PA . You are receiving this because you authored the thread.Message ID: @.***>

exalate-issue-sync[bot] commented 2 years ago

Adrian Damian commented: Every single move triggers an update in the DB. Depending on the number of updates that might take a while. I could also be just a blip in the system. We haven’t had similar complaints but can investigate further if this is a big issue. Please let me know.

Thanks,

Adrian