search

ubilabs / esa-climate-from-space

Climate from Space application for ESA's CCI+ program.
https://cfs.climate.esa.int
Other
9 stars 3 forks source link

Permafrost: update #408

Open pmast opened 4 years ago

pmast commented 4 years ago
pwambach commented 4 years ago

This is the one we had in the beginning. gdal didn't use the correct transparency values here so everything was either blue or completely transparent. The one we use now has just more steps in between

100 37 87 172 255
80 37 87 172 204
60 37 87 172 153
40 37 87 172 102
20 37 87 172 51
0 37 87 172 0
nv 0 0 0 0

Is something wrong with how it looks right now?

pmast commented 4 years ago

Looks much better! I will send this for review, then we will see!

pmast commented 3 years ago

Updated data set: Permafrost: /neodc/esacci/permafrost/data/permafrost_extent/L4/area4/pp/v02.0

pwambach commented 3 years ago

V02.0 has some strange behaviour. I tried out a lot of projection stuff but nothing works.

gdalinfo outputs:

polar_stereographic#crs_wkt=PROJCS["WGS 84 / Arctic Polar Stereographic",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]],PROJECTION["Polar_Stereographic"],PARAMETER["latitude_of_origin",71],PARAMETER["central_meridian",0],PARAMETER["scale_factor",1],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["X",EAST],AXIS["Y",NORTH],AUTHORITY["EPSG","3995"]]

but using gdalwarp -s_srs EPSG:3995 doesn't fit

pmast commented 3 years ago

Version 3.0 is available: https://catalogue.ceda.ac.uk/uuid/6e2091cb0c8b4106921b63cd5357c97c

pmast commented 3 years ago

Tried to generate tiles.

Logs: https://console.cloud.google.com/cloud-build/builds/d8415ccb-68bd-4885-85a6-b26136e014ab?project=612428468420

pwambach commented 3 years ago

Hmm, in step 5 there are no images files to copy and the step 2 which should produce the image files only took 32 seconds which seems way too short. But there is no error, so thats strange. You can reproduce the command from this step in the container on your local machine and try if that works. This is a bit of work because you have to call the command manually with the correct parameters. Before that I would test if the process works with lower zoom levels and/or lower resolution and a bigger machine type (e2-highcpu-32) to make sure that it's not a memory/performance issue because it's a huge dataset.

pwambach commented 3 years ago

The tile processing works in general but as with the former version there are again projection issues. I tried a lot of different settings but nothing worked. The best one was with using the projection definition from gdalinfo like this:

  gdalwarp \
    -s_srs "PROJCS["WGS_84_Arctic_Polar_Stereographic",GEOGCS["GCS_WGS_1984",DATUM["D_WGS_1984",SPHEROID["WGS_1984",6378137,298.257223563]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]],PROJECTION["Stereographic_North_Pole"],PARAMETER["standard_parallel_1",71],PARAMETER["central_meridian",0],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["Meter",1]]" \
    -t_srs EPSG:4326 \
    -te 360 -90 0 90 \
    -ts 2048 1024 \
    -r near \
    --config GDAL_CACHEMAX 90% \
    -co compress=LZW \
    NETCDF:\"./20100101.nc\":PFR \
    ./permafrost/tmp.tif

but this still results in a wrong image:

Screenshot 2021-08-09 at 16 36 22

Running out of ideas what to do here :(