search

FESOM / pyfesom2

FESOM2 tools https://pyfesom2.readthedocs.io
MIT License
16 stars 10 forks source link

Will this package work with CMIP6 data from the ESGF nodes? #130

Closed cspencerjones closed 2 years ago

cspencerjones commented 3 years ago

I would like to plot the AMOC from the AWI model CMIP6 data available via ESGF. It seems as though this package is not designed for that? Do you have any pointers about where to start?

koldunovn commented 3 years ago

I haven't tested it with CMORised FESOM data (most of the time we use native format, and pyfesom2 is for next generation of FESOM). It might work though, I will try to have a look.

@tsemmler05 I know you have computed AMOC recently from CMIP6 data, can you share some code for this?

@pgierz I remember you also had some experience with AMOC and FESOM1.4, can you, please, comment?

If you would need some other quantities and don't mind interpolation, this page might be useful: https://fesom.de/cmip6/work-with-awi-cm-unstructured-data/

pgierz commented 3 years ago

Hi,

I recently helped Tido compute AMOC with fesom 1.4 output. Is that the version submitted to ESGF?

I can provide that script, but it would need a decent clean first. @cspencerjones, if you're in a rush I can give you what we have right now, but it likely won't work out of the box

koldunovn commented 3 years ago

@pgierz thanks for the quick answer! There are 2 formats for 1.4 - with one long vector for 3d variables and with separate vector for each of the levels. The second one is the CMIP6 version.

cspencerjones commented 3 years ago

Thanks for your answers so far! @pgierz I am in a bit of a rush so yes I would love your script, if you are happy to share. Do you know which of the two formats it is for?

suvarchal commented 3 years ago

Another take: for simple spatial plotting, AFAIK, the mesh information that is needed is not included in ESGF (@pgierz correct me if i am wrong), If @koldunovn agrees I would like to address this using this issue. I think i makes so much sense to be able to use pyfesom2 to analyse CMIP6 AWI CM.

I am not sure if data can be added still to ESGF, regardless, here are options :

or do both.

I can quickly implement this if @koldunovn or @tsemmler05 or @pgierz help me figuring out meshes used for CMIP6

koldunovn commented 3 years ago

@suvarchal I think it makes total sense to be able to use pyfesom2 for CMIP6 data, and I know you already have some experience on this :)

It should not be too complicated as the data structures are similar except for location of vector variables and, I think also vertical placement of some variables. Certainly simple visualization should not be an issue. However some of the diagnostics can produce bad results (again due to different staggering), so we have to address that.

One way to do this is to only provide access to CMIP6 data through xarray accesor that you now creating (maybe do a special version for CMIP6), and implement diagnostics as methods one by one, making sure they work properly? One of the immediate problems is we don't have fesom.mesh.diag.

I don't think it makes sense to have meshes delivered with pyfesom2, I would vote for git lts on AWI gitlab (see below). Then we can do CMIP6 data notebook.

Meshes on GitLab: https://gitlab.awi.de/fesom1-meshes/fesom1_glob - HR https://gitlab.awi.de/fesom1-meshes/fesom1_core - LR

If Dirk did not opened the repositories yet, or if you want to double check here are the ones I used for CMIP6 data analysis (on mistral):

/mnt/lustre01/work/ab0995/a270088/meshes/CORE_AWICM_LR/core/

/mnt/lustre01/work/ab0995/a270088/meshes/glob/
suvarchal commented 3 years ago

@koldunovn thanks for the grids, i will take a look at grids and think of what can be done to at least support minimal capabilities like plotting and re-gridding to a structured grid. Then later think of how to use existing diagnostics, which i have no clue as of now.

suvarchal commented 3 years ago

Meshes on GitLab: https://gitlab.awi.de/fesom1-meshes/fesom1_glob - HR https://gitlab.awi.de/fesom1-meshes/fesom1_core - LR

is this MR? https://gitlab.awi.de/fesom1-meshes/fesom1_bold

koldunovn commented 3 years ago

Meshes on GitLab: https://gitlab.awi.de/fesom1-meshes/fesom1_glob - HR https://gitlab.awi.de/fesom1-meshes/fesom1_core - LR

is this MR? https://gitlab.awi.de/fesom1-meshes/fesom1_bold

Yes, it's MR, sorry :)

tsemmler05 commented 3 years ago

fesom1_core is LR fesom1_glob is MR fesom1_bold is HR

tsemmler05 commented 3 years ago

@pgierz : To compute AMOC from FESOM 1.4 which is the CMIP6 version I used your script. @suvarchal : it is still possible to publish data at ESGF.

helgegoessling commented 3 years ago

@koldunovn thanks for the grids, i will take a look at grids and think of what can be done to at least support minimal capabilities like plotting and re-gridding to a structured grid. Then later think of how to use existing diagnostics, which i have no clue as of now.

@suvarchal regarding the regridding to structured grids, you do know there's a way to do this with CDO using the "grid description files", as mentioned here (https://fesom.de/cmip6/work-with-awi-cm-unstructured-data/), right?

suvarchal commented 3 years ago

@pgierz : To compute AMOC from FESOM 1.4 which is the CMIP6 version I used your script. @suvarchal : it is still possible to publish data at ESGF.

Thanks Tido, I think few auxillary variables like faces of triangles would be great, areacello atleast for scalars would also be great. but faces, i think are most important without which it is not possible to do accurate spatial plots. Ahh, may be we take that conversation else where to FESOM? not sure if it is relevant here.

suvarchal commented 3 years ago

@koldunovn thanks for the grids, i will take a look at grids and think of what can be done to at least support minimal capabilities like plotting and re-gridding to a structured grid. Then later think of how to use existing diagnostics, which i have no clue as of now.

@suvarchal regarding the regridding to structured grids, you do know there's a way to do this with CDO using the "grid description files", as mentioned here (https://fesom.de/cmip6/work-with-awi-cm-unstructured-data/), right?

I heard about it (using unstructured grids as input) but never tried myself, i should do that.

tsemmler05 commented 3 years ago

Jan Hegewald has already prepared areacello, thckcello (thickness of each model layer), and deptho (bathymetry for each mesh point) in CMOR format and asked DKRZ for publication at ESGF. He has used the grid description files indeed. So no need to worry about these fixed fields anymore - just faces of triangles would be left to be done.

cspencerjones commented 3 years ago

Jan Hegewald has already prepared areacello, thckcello (thickness of each model layer), and deptho (bathymetry for each mesh point) in CMOR format and asked DKRZ for publication at ESGF. He has used the grid description files indeed. So no need to worry about these fixed fields anymore - just faces of triangles would be left to be done.

Ahh ok - it seems as though I need to wait for this anyway before doing anything. Thanks for the discussion: I'll come back to this when the fields are available.

koldunovn commented 2 years ago

Please reopen if problem is not solved.