search

EMCECS / ecs-sync

ecs-sync is a bulk copy utility that can move data between various systems in parallel
Apache License 2.0
60 stars 22 forks source link

Invalid Clip List #69

Closed aphoenix01 closed 2 years ago

aphoenix01 commented 4 years ago

I have generated a Centera Cliplist which is about 22 GB in size, I have subsequently broken this list down into multiple clip lists and distributed the jobs. When I verified the clips on ECS using Parallel-Dump, I get a list that is 8.9 GB in size. So then I create a difference file, break that up into smaller chunks and continue to Sync the clips.

I then performed another Parallel-Dump and get an 8.9 GB clip size, I then created another difference file and tried again. This time I get errors suggesting that the clip already exists and tries to force-sync to make sure both clips and BLOBS are copied.

2020-05-02 03:34:03 WARN [sync-pool-5-t-1] CasStorage: CAS clip 42N5TE6ABIJCIe6DC128SI9HJOHG413NT2RQKC07E7VE5A2M8GOU2 already exists; recopying the data due to force-sync option

I seem to be trapped in this endless loop, EMC have simply said the cliplist is invalid, but offer no further support about how its invalid and why I cant get a valid clip list to work from. They have also suggested a full validation check, but I have no idea how to perform this.

Does anyone have any suggestions about how I can perform a validation check?

aphoenix01 commented 4 years ago

I have been advised that the parrallel-dump utility is the cause of the problem and is creating the invalid cliplist, does anybody know how to generate a cliplist on ECS?

twincitiesguy commented 2 years ago

Closing due to inactivity.

In case anyone else stumbles upon this, Dell EMC Support can generate a clip list for customers. Unfortunately, I'm not aware of any customer-facing tool similar to parallel-bucket-dump.