search

EMCECS / ecs-sync

ecs-sync is a bulk copy utility that can move data between various systems in parallel
Apache License 2.0
61 stars 22 forks source link

Scan calculating never finish #110

Open mpmafra1403 opened 3 months ago

mpmafra1403 commented 3 months ago

Hi Folks, I'm migrating a S3 Object Storage bucket to a ECS, my job is running around 30 days, 2.3TB of data and 297M objects was copied and jobs still showing calculating, when I did CAS to CAS migrating this step was quickler, anything that I can do it to better perform this step? I need to give a estimative of time to customer and ecssync not help on that.

HCP to ECS - NFE - 19062024 (2019) ECSSYNC

dunedodo commented 3 months ago

Hi @mpmafra1403 you can have separate jobs on different nodes to migrate your bucket. This can reduce the "calculating" time to do the full scan. And please note that you can tune proper thread number for each job.