No extent field in YAML file for Sentinel-1 and Sentinel-2 data/images

[earthtech23]

Dear contributors,

When running this command: datacube-ows-update --schema --role admin We have this error:

Traceback (most recent call last):
  File "/usr/local/bin/datacube-ows-update", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/datacube_ows/update_ranges_impl.py", line 85, in main
    create_views(dc)
  File "/usr/local/lib/python3.10/dist-packages/datacube_ows/update_ranges_impl.py", line 126, in create_views
    run_sql(dc, "extent_views/create", database=dbname)
  File "/usr/local/lib/python3.10/dist-packages/datacube_ows/update_ranges_impl.py", line 183, in run_sql
    psycopg2connection.execute(q)
psycopg2.errors.InternalError_: parse error - invalid geometry
HINT:  "POLYGON((  , " <-- parse error at position 13 within geometry

The "solution" was to include the missing "extent" object in the YAML:

  "extent": {
    "lat": {
      "end": 35.9706782020955,
      "begin": 34.38661224451259
    },
    "lon": {
      "end": 35.024825846865966,
      "begin": 31.702062652092778
    }
  },

Why extent object is missing? Any ideas on fixing it?

Thanks in advance, Stelios

[SpacemanPaul]

How was your data indexed? Did you use stac-to-dc? If so from which STAC collection? Are you using a custom metadata type?

[earthtech23]

Our data are not indexed with stac-to-dc. I am using the "old" way by preparing dataset definitions according to the documentations and eo3 metadata type. With the same way i did it for 10 different MODIS products and i faced zero issues.

[SpacemanPaul]

See https://github.com/opendatacube/eo3/blob/develop/SPECIFICATION.md

Perhaps you are missing the $schema=https://schemas.opendatacube.org/dataset line?

[earthtech23]

Hey Paul, this was the first thing we thought about as an issue, but it's not missing!

Of course i have read the documentation, this is why, the missing extent made me curious.

[pierostz]

@earthtech23 and I are working together on this. We stumbled upon 2-3 confusing issues. This was one, another one is even though the no-data values type in our dataset was set to uint16, datacube-ows was complaining that the no-data values exceeded the uint8 value limits and another one which I cannot recall.

The solution to these issues was to update our datacube-core to the latest version 1.8.19. We were using 1.8.17. The reality is that I don't know if these issues were actual bugs coming in from datacube core or an incompatibility between the latest version of datacube-ows and an older version of datacube core.

In any case updating datacube-core was the solution.

[SpacemanPaul]

Great to hear you've got it working. Both OWS and Core are in active development at the moment. I'm not 100% sure what the underlying issue here was - perhaps downgrading OWS would have worked as well.