3D field analysis - questions

ocean-next / eNATL60

This repository describes eNATL60 simulations performed by Ocean Next and the MEOM group @ IGE in 2019.

23 stars 4 forks source link

3D field analysis - questions #3

Open apatlpo opened 3 years ago

apatlpo commented 3 years ago

Congrats for your effort here guys ! This is definitely the way to go !

We (@NoeLahaye, a student) could be interested in analyzing 3D fields with pangeo tools (dask distributed notably) but we would need a couple of information in order to decide whether this is the most sensible strategy:

we do not have accounts on active GENCI accounts. How could we remedy this issue? Would we have to submit a project to do the postprocessing? A positive answer to the latter question would strongly discourage our effort given the small amount of cpu time our analysis would take.
can we perform dask distributed postprocessing on gency clusters these days? Interactively?

apatlpo commented 3 years ago

ping @lesommer @brodeau @auraoupa

auraoupa commented 3 years ago

Hi @apatlpo and @NoeLahaye, thanks for your interest !

For eNATL60 3D fields they are permanently stored on occigen@CINES, so indeed you need a GENCI account to access it directly and perform some dask+xarray post processing on them (requesting CPU nodes via dask-jobqueue).

For the moment only small sub-datasets are available in 3D outside occigen, I can put them on hal@CNES if you like (or a personnalized sub-region) ?

Also, in the future some 3D data will be available on the cloud (VESRI-M2LINES project)

We can talk about it with @lesommer if needed !

apatlpo commented 3 years ago

thanks @auraoupa

Having access to sub-datasets would significantly limit the relevance of the effort for us.

We are also unfortunately constrained by time as we would need to start setting up computations in Feb/March for a student that would arrive in March I believe.

We do have an alternative with another similar modeling effort but I see more fruitful perspectives in the long run if we use eNATL60, hence my inquiries.

let's see what @lesommer has to say about this ...

lesommer commented 3 years ago

Hi @apatlpo, thanks for your interest in eNATL60 simulations ! We would be very happy to help you accessing the model data indeed. In your case, I think the simplest option would be for us to get you a GENCI account through our computing resources. We have some computing time secured for this sort of post processing. @auraoupa : could we sort this out by email ? My 2 cents

apatlpo commented 3 years ago

you're right, let's switch to emails ...