What happens to download requests?

[DamienIrving]

When you run compare_ESGF.py, it ends with a message asking if you'd like to submit a request to download the files. If you click yes, a text file containing a list of files appears in your current working directory and as far as I can tell a copy also appears at /g/data1/ua6/unofficial-ESG-replica/tmp/pxp581/requests/.

How does one find out when their request has been completed? (and is this the approach I should be taking to make download requests, or is there another preferred way?)

[paolap]

Eventually they will go to a share folder that any data downloader would be able to access. Then say someone from csiro will download data for you. We should have a better workflow, at the moment I'm looking after that but it was more as a trial to see how difficult could it be. I'm on my last day before a long leave, I was trying to update the db with enough bulk data transfer info so the searches on requests I received already would give a full picture of what's on raijin. When I'm sure the files aren't anywhere on raijin already I'm downloading them. Technically you're not ARCCSS so I'll have to look into the other users first but really I'll download your files too if I get there in time. I suppose in the future th euser would receive an e-mail when the request is actioned and then completed.

[DamienIrving]

Great, thanks @paolap. Who should I contact about data downloads while you're away? (i.e. the requests I've got in at the moment aren't a complete list of everything I'll eventually need)

[paolap]

That's the problem, Lawson has been downloading for BoM and Tim for CSIRO but it's not really clear and Tim is away too right now, Louise might leave too soon. It really something you should raise there

[DamienIrving]

Agreed. I'll raise it as an issue with Claire and Aurel. Enjoy your holiday!

[cet900]

Hi Damien, Unfortunately i don't really have an answer for you. Lawson might be able to schedule your downloads in with his, but they won't be merged with either the database or the DRS tree until Paola and Tim respectively are back on deck. I'm technically able to do downloads (and if you don't currently have an area you can write to I could easily set one up for you so you could too), but I'm run off my feet with other things, realistically I suspect you're going to have to either wait for Tim to return, or download yourself and worry about merging later? Sorry :-\ -Claire

[paolap]

I'm downloading the requests I've already received, to download stuff on your own you can use the requests files. Just download any wget script from the ESGF than substitute the list of files with the requests files themselves. Pretty much i have a script that concatanate the request in between two templates created from wget script. then you run the wget just discovered if there are no checksum listed use the -p option otherwise you go around in circles forever