When download full ds after 1 or two files, directory layout by rucio different

LHCAtlas / AtlasSSH

Commands to run Atlas software from a library via SSH to a linux node

MIT License

1 stars 1 forks source link

When download full ds after 1 or two files, directory layout by rucio different #47

Closed gordonwatts closed 9 years ago

gordonwatts commented 9 years ago

This causes problems with finding the dataset, and duplicate files. Is this us or is this rucio?

gordonwatts commented 9 years ago

Ok, according to DAST this just isn't easily possible. So, we will have to implement some sort of pruning I guess. When there are one or two files I don't think it matters. But if you download half the dataset first, and then the whole thing, that seems like wasted bandwidth. :(

gordonwatts commented 9 years ago

So, the toher thing to do is just only download the new files. This is the route I'm trying to take here.