Closed vkt1414 closed 8 months ago
Is it a good idea to have --show-progress here? I thought it is better to not use it in the manifest, and instead (only if needed) add it to s5cmd run.
I always use --show-progress in the notebooks as it is the only way to know how many slices were downloaded and at what speed.
To confirm I understand correctly: are you saying that passing --show-progress
to s5cmd run
has a different effect from passing this flag to every cp
command inside the manifest?
Is it a good idea to have --show-progress here? I thought it is better to not use it in the manifest, and instead (only if needed) add it to s5cmd run.
I always use --show-progress in the notebooks as it is the only way to know how many slices were downloaded and at what speed.
To confirm I understand correctly: are you saying that passing
--show-progress
tos5cmd run
has a different effect from passing this flag to everycp
command inside the manifest?
Sorry I did not mean any effect but as far I understand, unlike --endpoint-url
argument that can be set at the global level, we must pass --show-progress
with every cp command.
You are right! I thought I ran it with that flag passed to run
, but I was mistaken.
The previous query was removing localizers at the slice level while the expectation was to remove the entire series. The modified query filters the series containing localizer. In addition, the s5cmd urls are now generated from aws_urls.