UGRID reading issues (FESOM)

[koldunovn]

I am trying to make FESOM data work with QGIS and "Mesh Layer", and use version 3.8 of QGIS. AFAIU QGIS uses MDAL to read the data, so I open the issue here if I should instead open it in QGIS, pleas let me know.

The idea was to convert FESOM mesh to the same format as bw_11_zonder_riviergrid_met_1dwtg_map.nc with xarray. The only way I was able to make it work is by setting the type of the Mesh2D_face_nodes to float32, which is a bit weird since they are just indexes. I would appreciate if someone from your team could look at this issue.

Example files can be found at this folder: https://swiftbrowser.dkrz.de/public/dkrz_c719fbc3-98ea-446c-8e01-356dac22ed90/UGRID_MDAL/

fesom2_mesh_ugrid_not_working_nocycling.nc - one that can't be opened by QGIS with int type for Mesh2D_face_nodes

fesom2_mesh_ugrid_working_nocycling.nc - one that CAN be opened by QGIS with float32 type for Mesh2D_face_nodes

[PeterPetrik]

hi @koldunovn thanks for the bug report.

FESOM data is also netCDF but with some different convention than CF/UGRID? one way would be to support directly this data so you do not need any conversion from FESOM to something else...

for your question it is strange since we read it as int, see https://github.com/lutraconsulting/MDAL/blob/d44bd3e1f14ce51a6d9a5f8b40ef512b78274e1f/mdal/frmts/mdal_ugrid.cpp#L216

which MDAL version do you use?

[koldunovn]

Hi, @PeterPetrik thanks for the quick response!

Yes FESOM data are also in netCDF format, but we keep it extremely simple [time, number_of_nodes/number_of_faces, number_of_levels]. The mesh lives separately and consists of ASCII files with (i) coordinates of nodes, (ii) elements, (iii) vertical levels and depth at every node. We also have a netCDF description of the mesh, that lives separately. If you interested in implementing the format in MDAL, let me know I will be happy to contribute as much as I can.

I use MDAL version that comes with QGIS3.8 for Mac OS. Not shure how to check the MDAL version.

The strange thing is that for bw_11_zonder_riviergrid_met_1dwtg_map.nc the ncdump show int type for Mesh2D_face_nodes, but xarray reads it as float64. The xarray usually tries to preserve the types from original file.

It might be that the problem is related to my sistem after all, but would it be possible to make the code more independable of the type the variable?

[PeterPetrik]

System should not be a problem, I use MacOS too.

for reading of the data we use netcdf library which should take care of conversion of datatypes automatically. I assume the problem might be somewhere else. Best way is to compile debug version of mdalinfo and find out in debugger what is going on.

for the FESOM format, it looks like it is the layered 3D mesh UGRID type that is not yet supported but will be hopefully next year. Anyways we could read each level as separate dataset group for now. It is not problem that mesh is different file than datasets (we have precedence in 2dm reader and dat reader format). To implement FESOM it would take us 3 days of work. If you are interested in funding the work, please contact us on info@lutraconsulting.co.uk

Anyways, it would be good to have a small, public testcase for your automatic test suite with the format (ideally less than 3 MB).

[koldunovn]

Got it. I am in principle can live with connectivity information being float, and having the possibility to convert FESOM data to URGID is not a bad idea after all :)

Concerning the implementation of native FESOM format to MDAL, I am certainly interested and I will try to investigate if we can fund such work in principle first.

Thanks for looking at the issue!

[PeterPetrik]

no problem, another option is to add your FESOM->UGRID script to lutraconsulting/crayfish as new processing algorithm (similar to SAGA support)...

[saberraz]

@koldunovn we have implemented mdal_translate (similar to gdal_translate). It would be good to have your script incorporated into the mdal_translate utility, once we have FESOM support.