Data display

[lucapaganotti]

Hi all,

I'm new to open datacube and maybe this is not an issue at all, I installed the latest version of datacube-core, I've defined product and dataset metadata files (see below), added product and dataset to the datacube and then, in order to display the data I installed datacube-explorer. All seem to work but I can't see the data (see the image), I see only the data bounding box. For the time being I'm using a single stacked netCDF file for testing.

I see also the four bands I've defined for my data file, but I think that with great probability here I'm doing something wrong (see image).

I'm facing difficulties in writing product and dataset metadata files, I'm not able to find an extensive guide on how these files can be written, I found some examples but I spent a lot of time in finding the correct syntax for my use case, I'm using eo3.

Here is the product metadata yaml file:

name: c_o3_no_z
description: ARPA Ozone cover.
metadata_type: eo3

metadata:
  product:
    name: c_o3_no_z

load:
  crs: EPSG:32632
  resolution:
    x: 4000.0
    y: 4000.0
  align:
    x: 3000.0
    y: 1000.0

storage:
  crs: EPSG:32632
  resolution:
    x: 4000.0
    y: 4000.0

measurements:
  - name: "c_o3_01"
    aliases: [c_O3_01]
    dtype: float64
    nodata: -9999
    units: ppb
  - name: "c_o3_02"
    aliases: [c_O3_02]
    dtype: float64
    nodata: -9999
    units: ppb
  - name: "c_o3_03"
    aliases: [c_O3_03]
    dtype: float64
    nodata: -9999
    units: ppb
  - name: "c_o3_04"
    aliases: [c_O3_04]
    dtype: float64
    nodata: -9999
    units: ppb

And then the dataset yaml file:

# UUID of the dataset
id: 94cf3426-6200-44c8-b1b2-f4e37e850791
$schema: 'https://schemas.opendatacube.org/dataset'

# Product name
product: 
  name: c_o3_no_z

# Native CRS
crs: epsg:32632

# Optional GeoJSON object in the units of native CRS.
# Define a Polygon such that all valid pixels across all bands 
# are inside the polygon
geometry: 
  type: Polygon
  coordinates: [[[697000.0, 5175000.0], [697000.0, 4943000.0], [457000.0, 4943000.0], [457000.0, 5175000.0], [697000.0, 5175000.0]]]

grids:
  default:
    shape: [59, 61]
    transform: [4000.0, 0.0, 455000.0, 0.0, -4000.0, 5177000.0, 0.0, 0.0, 1.0]
    # spatial_reference: GEOGCS["GCS_WGS_1984",DATUM["WGS_1984",SPHEROID["WGS_84",6378137.0,298.257223563]],PRIMEM["Greenwich",0.0],UNIT["degree",0.0174532925199433],AUTHORITY["EPSG","4326"]]
    spatial_reference: PROJCS["WGS 84 / UTM zone 32N",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.01745329251994328,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]],UNIT["metre",1,AUTHORITY["EPSG","9001"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",9],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],AUTHORITY["EPSG","32632"],AXIS["Easting",EAST],AXIS["Northing",NORTH]]

measurements:
  c_o3_01:
    path: file:///home/buck/dev/odc/data/netcdf/test_no_zeta.nc#part=0
    # band: 1
    layer: c_O3
  c_o3_02:
    path: file:///home/buck/dev/odc/data/netcdf/test_no_zeta.nc#part=1
    # band: 2
    layer: c_O3
  c_o3_03:
    path: file:///home/buck/dev/odc/data/netcdf/test_no_zeta.nc#part=2
    # band: 3
    layer: c_O3
  c_o3_04:
    path: file:///home/buck/dev/odc/data/netcdf/test_no_zeta.nc#part=3
    # band: 4
    layer: c_O3

properties:
  dtr:start_datetime: 2015-04-10T02:00:00.000Z
  dtr:end_datetime: 2015-04-11T01:00:00.000Z
  eo:instrument: WAISALA
  eo:platform: arpa
  odc:file_format: NetCDF
  odc:processing_datetime: 2022-05-12T18:02:03.926659

lineage: {}

I would like to know if a detailed guide on eo3 format is available in order to study it. Then I would like to see my data displayed somewhere, I opted for the datacube-explorer that seems a lightweight solution in order to see quickly if I'm doing right. I found also the datacube OWS project that I think requires some more effort to be istalled and setup. If I need it please let me know.

Maybe I'm doing some trivial mistake, please be patient.

Have a good day and thank you for any answer..

[lucapaganotti]

Sorry, I forgot to ask also if there is the possibility to interface and/or query an open datacube instance via qgis desktop.

Thank you.

[uchchwhash]

I don't think you are doing anything particularly wrong, but explorer really only shows the dataset extents as you found out. If you want images, you want OWS. OWS also has an API to talk to QGIS, but it may not be as mature as you would like.

Feel free to ask any questions that you might have here.

[lucapaganotti]

Thank you @uchchwhash ,

in the meantime I tried to setup datacube-ows. I setup my environment, install it and then try

datacube-ows-update --schema --role <myrole>

I get these errors:

(odc) buck@poetic:~/dev/odc/datacube-ows/datacube_ows$ datacube-ows-update --schema --role odcusr
Traceback (most recent call last):
  File "/home/buck/anaconda3/envs/odc/bin/datacube-ows-update", line 5, in <module>
    from datacube_ows.update_ranges_impl import main
  File "/home/buck/anaconda3/envs/odc/lib/python3.8/site-packages/datacube_ows/update_ranges_impl.py", line 16, in <module>
    from datacube import Datacube
  File "/home/buck/dev/odc/datacube-core/datacube/__init__.py", line 29, in <module>
    from .api import Datacube
  File "/home/buck/dev/odc/datacube-core/datacube/api/__init__.py", line 9, in <module>
    from .core import Datacube, TerminateCurrentLoad
  File "/home/buck/dev/odc/datacube-core/datacube/api/core.py", line 16, in <module>
    from datacube.storage import reproject_and_fuse, BandInfo
  File "/home/buck/dev/odc/datacube-core/datacube/storage/__init__.py", line 11, in <module>
    from ..drivers.datasource import (
  File "/home/buck/dev/odc/datacube-core/datacube/drivers/__init__.py", line 9, in <module>
    from .indexes import index_driver_by_name, index_drivers
  File "/home/buck/dev/odc/datacube-core/datacube/drivers/indexes.py", line 9, in <module>
    from ..index.abstract import AbstractIndexDriver
  File "/home/buck/dev/odc/datacube-core/datacube/index/__init__.py", line 9, in <module>
    from ._api import index_connect
  File "/home/buck/dev/odc/datacube-core/datacube/index/_api.py", line 12, in <module>
    from datacube.index.abstract import AbstractIndex
  File "/home/buck/dev/odc/datacube-core/datacube/index/abstract.py", line 18, in <module>
    from datacube.index.fields import Field
  File "/home/buck/dev/odc/datacube-core/datacube/index/fields.py", line 13, in <module>
    from datacube.model import Range
  File "/home/buck/dev/odc/datacube-core/datacube/model/__init__.py", line 20, in <module>
    from datacube.utils import geometry, without_lineage_sources, parse_time, cached_property, uri_to_local_path, \
  File "/home/buck/dev/odc/datacube-core/datacube/utils/geometry/__init__.py", line 8, in <module>
    from ._base import (
  File "/home/buck/dev/odc/datacube-core/datacube/utils/geometry/_base.py", line 19, in <module>
    import rasterio                    # type: ignore[import]
  File "/home/buck/anaconda3/envs/odc/lib/python3.8/site-packages/rasterio/__init__.py", line 28, in <module>
    from rasterio._version import gdal_version, get_geos_version, get_proj_version
ImportError: libLerc.so.4: cannot open shared object file: No such file or directory

stating that I'm basicly missing libLerc dynamic library, I've tried to search for it in my distribution packages and in the conda environment but I'm not able to find out what package I do need to install.

I'm running all on a debian 11 box updated up today, but package lerc or liblerc seems not to be available. Should I use another distribution or get the package from an ubuntu repository that seems to have it?

It seems that libeLerc is a rasterio related package but was marked as unstable by debian distributions starting from liblerc3 in sid distribution.

I'm not sure that I have to ask here, matbe I shoul ask for help @ datacube-ows project.

Any help will be appreciated.

Thanks again for your answer.

[uchchwhash]

I am not really sure about this error message, I have not encountered it before. Getting the environment working properly is a tricky issue for ODC or Python in general, especially when pre-compiled binaries are involved.

Can I suggest a clean installation of ODC in an isolated conda environment? Another option could be using Docker.

[alexgleith]

Doing sudo apt-get -y install liblerc-dev should fix the library dependency.

On actual data previews, it would be nice to at least show the RBG thumbnails if they exist, but there's no way for the ODC to flag that these exist in a general way. Even nicer would be flagging the fact an OWS instance exists and provide renders from that, but again, they're not really closely related currently, Explorer and OWS.

[SpacemanPaul]

This appears to now be a datacube-ows question rather than an Explorer issue - you could raise an issue on the datacube-ows repository, or for this sort of new-starter initial-setup help questions, you might be better off joining the OpenDataCube Slack and asking on the #ows channel.

[uchchwhash]

I think in some sense the issue is resolved now. So closing. Please re-open if you disagree.