search

Reading-eScience-Centre / ncwms

ncWMS - A Web Map Service for displaying environmental data over the web
Other
62 stars 30 forks source link

Cordex dataset visualization issues #93

Closed moovida closed 2 years ago

moovida commented 2 years ago

Hello, I am not sure if this is more a question or an issue. Trying to describe it.

I got a couple of netcdf cordex meteo datasets of various cordex domains: https://cordex.org/data-access/regional-climate-change-simulations-for-cordex-domains/

While for example the European region looks ok to me:

image

the African is placed between Australia and America:

image

But then again another European gets misplaced (you can see Italy's boot shifted east:

image

Is there any way to make ncWMS understand what is going on with the projections? In the above examples I see two rotated pole projections with diverse parameters and one lambert conformal. But only the first is properly identified.

Thanks for any hint you can give me.

PS: For completeness I report the relevant information about the projection and metadata that gdal is able to read out of the datasets.

Europe Right:

Origin = (-28.430000000000000,21.890000000000001)
Pixel Size = (0.110000000000000,-0.110000000000000)
Metadata:
  NC_GLOBAL#Conventions=CF-1.4
  NC_GLOBAL#CORDEX_domain=EUR-11
  NC_GLOBAL#creation_date=2014-06-22-T10:38:52Z
  NC_GLOBAL#driving_experiment=IPSL-IPSL-CM5A-MR, rcp45, r1i1p1
  NC_GLOBAL#driving_experiment_name=rcp45
  NC_GLOBAL#driving_model_ensemble_member=r1i1p1
  NC_GLOBAL#driving_model_id=IPSL-IPSL-CM5A-MR
  NC_GLOBAL#experiment=RCP4.5
  NC_GLOBAL#experiment_id=rcp45
  NC_GLOBAL#frequency=day
  NC_GLOBAL#institute_id=SMHI
  NC_GLOBAL#institution=Swedish Meteorological and Hydrological Institute, Rossby Centre
  NC_GLOBAL#model_id=SMHI-RCA4
  NC_GLOBAL#product=output
  NC_GLOBAL#project_id=CORDEX
  NC_GLOBAL#rcm_version_id=v1
  NC_GLOBAL#references=http://www.smhi.se/en/Research/Research-departments/climate-research-rossby-centre
  NC_GLOBAL#rossby_comment=201403: CORDEX Europe 0.11 deg | RCA4 v1 | IPSL-IPSL-CM5A-MR | r1i1p1 | rcp45 | L40
  pr#coordinates=lon lat
  pr#grid_mapping=rotated_pole
  pr#long_name=Precipitation
  pr#missing_value=1e+20
  pr#standard_name=precipitation_flux
  pr#units=kg m-2 s-1
  pr#_FillValue=1e+20
  rlat#axis=Y
  rlat#long_name=latitude in rotated pole grid
  rlat#standard_name=grid_latitude
  rlat#units=degrees
  rlon#axis=X
  rlon#long_name=longitude in rotated pole grid
  rlon#standard_name=grid_longitude
  rlon#units=degrees
  rotated_pole#grid_mapping_name=rotated_latitude_longitude
  rotated_pole#grid_north_pole_latitude=39.25
  rotated_pole#grid_north_pole_longitude=-162
Geolocation:
  LINE_OFFSET=0
  LINE_STEP=1
  PIXEL_OFFSET=0
  PIXEL_STEP=1
  SRS=GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AXIS["Latitude",NORTH],AXIS["Longitude",EAST],AUTHORITY["EPSG","4326"]]

Africa wrong:

Origin = (-24.860000018925628,42.460002290340434)
Pixel Size = (0.220000037851260,-0.220000003044147)
Metadata:
  NC_GLOBAL#Conventions=CF-1.4
  NC_GLOBAL#CORDEX_domain=AFR-22
  NC_GLOBAL#creation_date=2012-12-22-T23:17:00Z
  NC_GLOBAL#driving_experiment=CCCma-CanESM2, rcp85, r1i1p1
  NC_GLOBAL#driving_experiment_name=rcp85
  NC_GLOBAL#driving_model_ensemble_member=r1i1p1
  NC_GLOBAL#driving_model_id=CCCma-CanESM2
  NC_GLOBAL#experiment=RCP8.5 run driven by CCCma-CanESM2 
  NC_GLOBAL#experiment_id=rcp85
  NC_GLOBAL#forcing=GHG,Oz,SA,BC,OC,LU,Vl (GHG includes CO2,CH4,N2O,CFC11,effective CFC12)
  NC_GLOBAL#frequency=day
  NC_GLOBAL#history=created: 2012-12-22 23:33:20 by rcm2nc
  NC_GLOBAL#institute_id=CCCma
  NC_GLOBAL#institution=CCCma (Canadian Centre for Climate Modelling and Analysis, Victoria, BC, Canada)
  NC_GLOBAL#model_id=CCCma-CanRCM4
  NC_GLOBAL#product=output
  NC_GLOBAL#project_id=CORDEX
  NC_GLOBAL#rcm_version_id=r2
  NC_GLOBAL#references=http://www.cccma.ec.gc.ca/models
  NC_GLOBAL#title=CanRCM4 model output prepared for CORDEX Project
  NC_GLOBAL#tracking_id=hdl:21.14103/7b606595-03af-47d9-90a5-75f88864421b
  rlat#axis=Y
  rlat#long_name=latitude in rotated pole grid
  rlat#standard_name=grid_latitude
  rlat#units=degrees
  rlon#axis=X
  rlon#long_name=longitude in rotated pole grid
  rlon#standard_name=grid_longitude
  rlon#units=degrees
  rotated_pole#grid_mapping_name=rotated_latitude_longitude
  rotated_pole#grid_north_pole_latitude=90
  rotated_pole#grid_north_pole_longitude=180
  tas#cell_methods=time: mean
  tas#coordinates=lon lat height
  tas#grid_mapping=rotated_pole
  tas#long_name=Near-Surface Air Temperature
  tas#missing_value=1e+20
  tas#standard_name=air_temperature
  tas#units=K
  tas#_FillValue=1e+20
Geolocation:
  LINE_OFFSET=0
  LINE_STEP=1
  PIXEL_OFFSET=0
  PIXEL_STEP=1
  SRS=GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AXIS["Latitude",NORTH],AXIS["Longitude",EAST],AUTHORITY["EPSG","4326"]]

Europe wrong:

Coordinate System is:
PROJCRS["unnamed",
    BASEGEOGCRS["WGS 84",
        DATUM["World Geodetic System 1984",
            ELLIPSOID["WGS 84",6378137,298.257223563,
                LENGTHUNIT["metre",1]]],
        PRIMEM["Greenwich",0,
            ANGLEUNIT["degree",0.0174532925199433]],
        ID["EPSG",4326]],
    CONVERSION["unnamed",
        METHOD["Lambert Conic Conformal (1SP)",
            ID["EPSG",9801]],
        PARAMETER["Latitude of natural origin",49.5,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8801]],
        PARAMETER["Longitude of natural origin",10.5,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8802]],
        PARAMETER["Scale factor at natural origin",1,
            SCALEUNIT["unity",1],
            ID["EPSG",8805]],
        PARAMETER["False easting",0,
            LENGTHUNIT["kilometre",1000],
            ID["EPSG",8806]],
        PARAMETER["False northing",0,
            LENGTHUNIT["kilometre",1000],
            ID["EPSG",8807]],
        PARAMETER["standard_parallel_1",49.5,
            ANGLEUNIT["degree",0.0174532925199433,
                ID["EPSG",9122]]]],
    CS[Cartesian,2],
        AXIS["easting",east,
            ORDER[1],
            LENGTHUNIT["kilometre",1000,
                ID["EPSG",9036]]],
        AXIS["northing",north,
            ORDER[2],
            LENGTHUNIT["kilometre",1000,
                ID["EPSG",9036]]]]
Data axis to CRS axis mapping: 1,2
Origin = (-6.250000000000000,5656.250000000000000)
Pixel Size = (12.500000000000000,-12.500000000000000)
Metadata:
  Lambert_Conformal#grid_mapping_name=lambert_conformal_conic
  Lambert_Conformal#latitude_of_projection_origin=49.5
  Lambert_Conformal#longitude_of_central_meridian=10.5
  Lambert_Conformal#standard_parallel=49.5
 NC_GLOBAL#Convention=CF-1.6
  NC_GLOBAL#Conventions=CF-1.6
  NC_GLOBAL#CORDEX_domain=EUR-11
  NC_GLOBAL#creation_date=2018-12-13T00:01:57Z
  NC_GLOBAL#description=Created by xios
  NC_GLOBAL#driving_experiment=CNRM-CERFACS-CNRM-CM5, rcp85, r1i1p1
  NC_GLOBAL#driving_experiment_name=rcp85
  NC_GLOBAL#driving_model_ensemble_member=r1i1p1
  NC_GLOBAL#driving_model_id=CNRM-CERFACS-CNRM-CM5
  NC_GLOBAL#experiment=RCP8.5 run with GCM forcing
  NC_GLOBAL#experiment_id=rcp85
  NC_GLOBAL#frequency=day
  NC_GLOBAL#institute_id=CNRM
  NC_GLOBAL#institution=CNRM (Centre National de Recherches Meteorologiques, Toulouse 31057, France)
  NC_GLOBAL#model_id=CNRM-ALADIN63
  NC_GLOBAL#name=EUR-11_CNRM-CERFACS-CNRM-CM5_rcp85_r1i1p1_CNRM-ALADIN63_v2_day
  NC_GLOBAL#product=output
  NC_GLOBAL#project_id=CORDEX
  NC_GLOBAL#rcm_version_id=v2
  NC_GLOBAL#references=http://www.umr-cnrm.fr/spip.php?article125&lang=en
 tas#cell_methods=time: mean
  tas#coordinates=lon lat height
  tas#grid_mapping=Lambert_Conformal
  tas#interval_operation=450 s
  tas#interval_write=1 d
  tas#long_name=Near-Surface Air Temperature
  tas#missing_value=1e+20
  tas#online_operation=average
  tas#standard_name=air_temperature
  tas#units=K
  tas#_FillValue=1e+20
Geolocation:
  LINE_OFFSET=0
  LINE_STEP=1
  PIXEL_OFFSET=0
  PIXEL_STEP=1
  SRS=GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AXIS["Latitude",NORTH],AXIS["Longitude",EAST],AUTHORITY["EPSG","4326"]]
guygriffiths commented 2 years ago

The way which ncWMS handles grids like this is via 2d lat-lon coordinate variables, as per https://cfconventions.org/Data/cf-conventions/cf-conventions-1.9/cf-conventions.html#_two_dimensional_latitude_longitude_coordinate_variables

Notably it completely ignores any other definition of the coordinate system.

We use Unidata's NetCDF-Java libraries for doing this projection, and it's a very stable longstanding part of the system, so I would guess that the data you are using doesn't define its coordinate variables in this standard way (which is not especially uncommon - many data providers use ad hoc mechanisms which supply all of the necessary data, just not in a way which follows the CF conventions).

If you can send me a sample dataset I'd be happy to take a look and see if this is the case. If so the only practical solution is to rewrite your coordinate variables so that they refer to the correct area.

moovida commented 2 years ago

Hy @guygriffiths , thanks for you reply. I though it might be as you describe it. I tried some other applications (panolpy for example) and they have the exact same issue, I guess the netcdf java libs just do not read these non standard ways of defining the prj (as you can se above, some have a WKT, others just the name).

If I can, I would love to send you the samples (placing the link here below, since they are quite big) and hear your best practice on how to solve it to make it work properly. I am not sure if you mean that one could insert some sort of plugin to ncWMS to tweak the metadata or if it is more about a preprocessing of the data. Either way, thank you.

Link to the data: https://we.tl/t-cOgpRXJhWU

guygriffiths commented 2 years ago

Thanks for sending me that. I think I've identified what the issue is. The coordinates are actually defined correctly, in the 2d coordinate variable lat and lon. The data is also correctly identifying these as the coordinates to use via the coordinates attribute. However, because your dimensions are named rlat/rlon and you also have these defined as variables with the units degrees, they are being seen as coordinate variables and used in preference to the 2d versions.

The way to fix it is to make sure that rlat and rlon are not coordinate variables, but just standard ones. That means they need to have a different name to the dimensions. If you have the NetCDF Operators, you can rename them using this command:

ncrename -v rlon,rotatedlon -v rlat,rotatedlat file.nc

If you prefer not to modify the data, you can use NcML, a markup language which ncWMS supports, to wrap your data files and rename the variables without touching the data itself. Create a file named filename.ncml in the same directory with the contents:

<?xml version="1.0" encoding="UTF-8"?>
<netcdf location="filename.nc"  
        xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2">
  <variable name="rotatedlon" orgName="rlon" type="float"/>
  <variable name="rotatedlat" orgName="rlat" type="float"/>
</netcdf>

(you will need to change the line containing filename.nc, obviously)

Then you can simply point ncWMS at this file instead of the original data file and it will work.

Note that I've actually only tried this with one dataset - the african one - because I'm currently WFH and my internet connection is pretty slow. It worked for that though, so I imagine the problem is the same for all of the datasets. Let me know how you get on.

moovida commented 2 years ago

Dear @guygriffiths , thanks a ton for the fast reply and sorry for my late reply, been knocked out by a flue. I never used the netcdf operator, so that will be my first choice to try to set the datasets straight. Great!

moovida commented 2 years ago

Actually compilation of the operators seems more tricky than expected. And I have to admit that the ncml trick is just great. And does work. Thank you for bearing with me and helping even if, obviously, it was not a problem of ncWMS. I will close the issue.