search

ESPRI-Mod / synda

ESGF Downloader (this is a deprecated repository, the tool has now moved to https://github.com/ESGF/esgf-download)
https://espri-mod.github.io/synda/
21 stars 11 forks source link

General Fallbacks #87

Open painter1 opened 6 years ago

painter1 commented 6 years ago

This is not a bug, it's just something on my wish list.

When a GridFTP download fails, Synda can fall back to http. I have encountered a situation where the data node fails, but I know of another one which has the same data. I would like to have Synda fall back to the other node. The other node would generally require a different path. So here's how I would want to specify the other node: I would provide a function. The only argument is the url which failed. My function would return another url to use instead.

Note that this interface would allow not only changing data nodes, but also changing protocols, e.g. GridFTP to http or others if supported by Synda.

painter1 commented 6 years ago

I have implemented a more general fallback method, with a new version of sdnexturl.py. It's not flexible as described in my request, but good enough for my needs. Priorities among data nodes are hard-wired; ideally that would be done in the configuration file instead. Let me know if you want to see it.

SebastienDenvil commented 6 years ago

Thanks Jeffrey. We will be happy to have a look. You could make a pull request so that we can have an idea.

painter1 commented 6 years ago

It looks as though I need write permission before I can push a branch. Could you set me up as a collaborator? Thank you!