Open-EO / openeo-geotrellis-extensions

Java/Scala extensions for Geotrellis, for use with OpenEO GeoPySpark backend.
Apache License 2.0
5 stars 3 forks source link

BioPar_NDVI_V2_Global. "... is a netCDF file, but ndvi is not a variable" #187

Closed EmileSonneveld closed 1 year ago

EmileSonneveld commented 1 year ago

This error occurs when running the following graph:

{
  "process_graph": {
    "loadcollection1": {
      "process_id": "load_collection",
      "arguments": {
        "bands": [
          "ndvi"
        ],
        "id": "CGLS_NDVI_V2_GLOBAL",
        "spatial_extent": {
          "west": 5.03,
          "south": 51.2,
          "east": 5.05,
          "north": 51.22,
          "crs": "EPSG:4326"
        },
        "temporal_extent": [
          "2020-07-01",
          "2020-08-01"
        ]
      }
    },
    "reducedimension1": {
      "process_id": "reduce_dimension",
      "arguments": {
        "data": {
          "from_node": "loadcollection1"
        },
        "dimension": "t",
        "reducer": {
          "process_graph": {
            "mean1": {
              "process_id": "mean",
              "arguments": {
                "data": {
                  "from_parameter": "data"
                }
              },
              "result": true
            }
          }
        }
      }
    },
    "saveresult1": {
      "process_id": "save_result",
      "arguments": {
        "data": {
          "from_node": "reducedimension1"
        },
        "format": "GTiff",
        "options": {}
      },
      "result": true
    }
  }
}

Example run: j-950834fff6144549a5fbf520ec001ac1

[1 of 1000] FAILURE(3) CPLE_AppDefined(1) "Application defined error." netcdf error #-49 : NetCDF: Variable not found .
at (netcdfdataset.cpp,NCDFOpenSubDataset,11446)

[2 of 1000] WARNING(2) CPLE_AppDefined(1) "Application defined error." NETCDF:"/data/MTDA/BIOPAR/BioPar_NDVI_V2_Global/2020/20200701/c_gls_NDVI_202007010000_GLOBE_PROBAV_V2.2.1/c_gls_NDVI_202007010000_GLOBE_PROBAV_V2.2.1.nc":ndvi is a netCDF file, but ndvi is not a variable. 
[3 of 1000] FAILURE(3) CPLE_OpenFailed(4) "Open failed." NETCDF:"/data/MTDA/BIOPAR/BioPar_NDVI_V2_Global/2020/20200701/c_gls_NDVI_202007010000_GLOBE_PROBAV_V2.2.1/c_gls_NDVI_202007010000_GLOBE_PROBAV_V2.2.1.nc":ndvi: No such file or directory 
[4 of 1000] FAILURE(3) CPLE_AppDefined(1) "Application defined error." netcdf error #-49 : NetCDF: Variable not found .
at (netcdfdataset.cpp,NCDFOpenSubDataset,11446)

[5 of 1000] WARNING(2) CPLE_AppDefined(1) "Application defined error." NETCDF:"/data/MTDA/BIOPAR/BioPar_NDVI_V2_Global/2020/20200701/c_gls_NDVI_202007010000_GLOBE_PROBAV_V2.2.1/c_gls_NDVI_202007010000_GLOBE_PROBAV_V2.2.1.nc":ndvi is a netCDF file, but ndvi is not a variable. 
[6 of 1000] FAILURE(3) CPLE_OpenFailed(4) "Open failed." NETCDF:"/data/MTDA/BIOPAR/BioPar_NDVI_V2_Global/2020/20200701/c_gls_NDVI_202007010000_GLOBE_PROBAV_V2.2.1/c_gls_NDVI_202007010000_GLOBE_PROBAV_V2.2.1.nc":ndvi: No such file or directory 

The job ends ends with this error: OpenEO batch job failed: geotrellis.raster.gdal.MalformedDataException: Unable to construct dataset dimensions. GDAL Error Code: 4

Issue found by Marcel

jdries commented 1 year ago

correct name is NDVI, so uppercase, may be fixed via simple update to layercatalog.json

EmileSonneveld commented 1 year ago

That indeed solves the issue. At least when running locally. @jdries, Can you review the pr? https://git.vito.be/projects/BIGGEO/repos/openeo-deploy/pull-requests/8/overview

EmileSonneveld commented 1 year ago

I did a query that covered 1998-2023 and got no errors

EmileSonneveld commented 1 year ago

In the future it would be nice to add this to the collection tests.