Investigate CDSE data access via STAC and S3

clausmichele commented 1 week ago

It would be interesting to show how to access data offered by the Copernicus Data Space in different ways. CDSE offers also a STAC Catalog, but it doesn't include the actual links to the files.

We need to investigate if, querying the STAC Catalog, it's possible to find the corresponding file links which are on S3. CDSE STAC docs: https://documentation.dataspace.copernicus.eu/APIs/STAC.html

Sample query to CDSE STAC:

from pystac_client import Client

catalog = Client.open('https://catalogue.dataspace.copernicus.eu/stac/')
items = catalog.search(
    collections=["SENTINEL-2"],
    bbox=[11.02083333, 46.65359938, 11.36666667, 46.95416667],
    datetime=['2018-02-01T00:00:00Z', '2018-06-30T00:00:00Z'],
).item_collection()
items

Independently from the previous result, we should show how to open (lazily) in Xarray one or more files from S3, using fsspec and/or s3fs. Documentation of S3 offered by CDSE: https://documentation.dataspace.copernicus.eu/APIs/S3.html

jzvolensky commented 6 days ago

I have created some example access and made some notes about my experience. Please see below and let me know how to proceed:

https://github.com/jzvolensky/cdse-stac-api/blob/master/cdse-stac-api/cdse_data_access.ipynb

clausmichele commented 6 days ago

Hi @jzvolensky, great that you found a way to get access to the data from STAC. Could you expand the Sentinel-2 example trying to open a file in the GRANULE folder available at eodata/Sentinel-2/MSI/L1C/2022/02/05/S2B_MSIL1C_20220205T102119_N0400_R065_T32TPT_20220205T110638.SAFE/GRANULE ?

jzvolensky commented 5 days ago

Getting access denied issues for the granule. Not sure if this is the fault of CDSE or my fsspec setup

fs = fsspec.filesystem('s3', key=S3_KEY, secret=S3_SECRET, client_kwargs={'endpoint_url': 'https://eodata.dataspace.copernicus.eu'})

granule_path = 's3://eodata/Sentinel-2/MSI/L1C/2022/02/05/S2B_MSIL1C_20220205T102119_N0400_R065_T32TPT_20220205T110638.SAFE/GRANULE/L1C_T32TPT_A025688_20220205T102141/IMG_DATA/T32TPT_20220205T102119_B04.jp2'

with rasterio.open(granule_path, opener=fs.open) as src:
    band4 = src.read(1)

band4_da = xarray.DataArray(band4, dims=["y", "x"])

print(band4_da)

EO-College / cubes-and-clouds

Investigate CDSE data access via STAC and S3 #53