Adding Landsat-8 Collection 2

[maximlamare]

Overview

In the context of the Africa Cube contest we need to give access to Landsat Collection-2 data (L1C and L2A) to participants. It would also be good for users to access these new datasets in EDC.

Current situation

In the current version of EDC, the collections are visible:

from xcube_sh.sentinelhub import SentinelHub
SH = SentinelHub()
SH.dataset_names

results in

['L8L1C',
 'S2L2A',
 'S1GRD',
 'CUSTOM',
 'LOTL2',
 'S2L1C',
 'DEM',
 'LOTL1',
 'MODIS',
 'S3OLCI',
 'S5PL2',
 'S3SLSTR']

However, the endpoint is different for Landsat-8 Collection 2: services-uswest2.sentinel-hub.com (see https://docs.sentinel-hub.com/api/latest/data/landsat-8-l2/#endpoint-locations).

So out of the box, the following commands don't work:

SH.band_names('LOTL1')
SH.band_names('LOTL2')

If one changes the endpoint, then the band names can be listed:

SH.api_url = ("https://services-uswest2.sentinel-hub.com")
SH.band_names('LOTL2')

results in:

['B01',
 'B02',
 'B03',
 'B04',
 'B05',
 'B06',
 'B07',
 'B10',
 'BQA',
 'QA_RADSAT',
 'SR_QA_AEROSOL',
 'ST_QA',
 'ST_TRAD',
 'ST_URAD',
 'ST_DRAD',
 'ST_ATRAN',
 'ST_EMIS',
 'ST_EMSD',
 'ST_CDIST']

Problem

When creating an xcube, for example:

cube_config = CubeConfig(dataset_name='LOTL2',
                         band_names=['B02', 'B03'],
                         bbox=dakar_bbox,
                         spatial_res=0.000089,
                         time_range=['2018-05-14', '2020-07-31'],
                         time_tolerance='30M')

We cannot open the cube with the normal endpoint:

cube = open_cube(cube_config)
cube

The code above generates the following error: ValueError: cannot find collection name for dataset name 'LOTL2'.

Now looking at the code, if we try to pass the SentinelHub object with the updated api_url as an argument of the open_cube function:

cube = open_cube(cube_config, sentinel_hub=SH)
cube

The above does not work either: ValueError: cannot find collection name for dataset name 'LOTL2'.

Solution tried

The problem comes from the data missing from the metadata. I added the missing metadata to the file, see MR coming up.

There are elements missing to my MR, because of the differences in endpoints. With the current state, when I call the cube with the new endpoint (https://services-uswest2.sentinel-hub.com), this causes a failure in the catalog:

SentinelHubError: 404 Client Error: Not Found for url: https://services-uswest2.sentinel-hub.com/api/v1/catalog/search

And of course, if I call the cube with the original endpoint, I get Collection not found.

Help needed

Can I get help to implement Landsat Collection 2 in time for the Africa contest (end of April)?

[forman]

@maximlamare the problem seems to originate from the Sentinel Hub catalogue. We'll need to find the corresponding catalogue entry for 'LOTL2'.

[maximlamare]

@forman I asked around, and have been given the entries for the catalogue:

The ids are landsat-ot-l1 and landsat-ot-l2

[forman]

The ids are landsat-ot-l1 and landsat-ot-l2

Great, I'll change that accordingly.

[AliceBalfanz]

Hi @maximlamare, changes have been merged to the xcube-sh master, could you test it, please? :)

from xcube_sh.sentinelhub import SentinelHub
from xcube_sh.config import CubeConfig
from xcube_sh.cube import open_cube

SH = SentinelHub()
SH.api_url = ("https://services-uswest2.sentinel-hub.com")

dakar_bbox = [-17.554176,14.640112,-17.387367,14.792487]

cube_config = CubeConfig(dataset_name='LOTL1',
                         band_names=['B02', 'B03'],
                         bbox=dakar_bbox,
                         spatial_res=0.000089,
                         time_range=['2018-05-14', '2020-07-31'],
                         time_tolerance='30M')

cube = open_cube(cube_config,sentinel_hub=SH)
cube

[maximlamare]

@AliceBalfanz that's awesome!

Do I need to update something in my EOXHub? Because I'm getting this when I run the above:

ValueError: cannot find collection name for dataset name 'LOTL1'

[AliceBalfanz]

Hi @maximlamare ,

if I understand correctly, we need to make a conda forge release which includes the changes and then ask EOX to make an update, otherwise you will not be able to use the new changes via the EOXhub.

Please correct, if I am wrong @dzelge

[forman]

@maximlamare xcube-sh 0.8.0 will be released on Wednesday. If you are using the EOX NBs, you will have to wait for another xcube-sh docker image that will inlcude the 0.8.0. This will still take another one or two weeks from then. But you can checkout and install latest xcube-sh from sources. In your Juypter lab open a terminal then

$ conda activate xcube
$ git clone https://github.com/dcs4cop/xcube-sh.git
$ cd xcube-sh
$ python setup.py develop
$ python

Verify:

>>> from xcube_sh.version import version
>>> version

[forman]

But you can checkout and install latest xcube-sh from sources. In your Juypter lab open a terminal then

No wait. In order to make that work, you'll also need to update xcube to latest version using same method.