search

EMCECS / ecs-sync

ecs-sync is a bulk copy utility that can move data between various systems in parallel
Apache License 2.0
61 stars 22 forks source link

Combination of verify and preserve-directories false throws Exception #87

Closed mzinsmeister closed 3 years ago

mzinsmeister commented 3 years ago

When using ECS-Sync with a filesystem source and an ECS-S3 target with the setting "preserve-directories" set to false on a source directory with subdirectories and verify turned on i get an 

2021-10-27 09:04:25 WARN  [sync-pool-1-t-8] SyncTask: O--! object /test/1 failed
com.emc.ecs.sync.storage.ObjectNotFoundException: 1/
        at com.emc.ecs.sync.storage.s3.EcsS3Storage.loadObject(EcsS3Storage.java:252)
        at com.emc.ecs.sync.storage.s3.AbstractS3Storage.loadObject(AbstractS3Storage.java:65)
        at com.emc.ecs.sync.storage.s3.EcsS3Storage.loadObject(EcsS3Storage.java:237)
        at com.emc.ecs.sync.TargetFilter.reverseFilter(TargetFilter.java:124)
        at com.emc.ecs.sync.SyncTask.run(SyncTask.java:128)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

Exception printed for each subdirectory. When using preserve-directories true i don't get this Exception. I'm pretty sure this is because the preserve-directories option is not used for verifying. When this option is false, the validation step should obviously be skipped. I'm not sure the Exception has any consequences except for the log entry though.

mzinsmeister commented 3 years ago

Ah, i just saw this is basically a duplicate of #44. Still kinda weird that an exception in the logs is the expected behaviour for this...