GeoscienceAustralia / dea-notebooks

Repository for Digital Earth Australia Jupyter Notebooks: tools and workflows for geospatial analysis with Open Data Cube and Xarray
https://docs.dea.ga.gov.au/notebooks/
Apache License 2.0
442 stars 128 forks source link

Planetary computer (PC) notebook suggestion for subscription only products #1130

Closed wwoodgate closed 1 year ago

wwoodgate commented 1 year ago

Hi DEA team,

First, I love the addition of the Planetary computer notebook. However, some PC products require an API subscription key to open (more permissioned).

If you could provide some suggestions or code as to how to implement this in the current notebook (if feasible) that would be fantastic.

For example, accessing S1 radar RTC data would be ace (https://planetarycomputer.microsoft.com/dataset/sentinel-1-rtc ). Here's a discussion that may be relevant (https://github.com/microsoft/PlanetaryComputer/discussions/167) - I hit a dead end there even though I have a PC account and API key..

Cheers, Will

robbibt commented 1 year ago

Hey @wwoodgate! Yeah, I have to admit I chose products for this notebook that didn't require a key to avoid this whole issue... 😅 It's a great idea though - some guidance on accessing data that requires an API key would be really useful to include here. Will have a think and see what we can work out!

robbibt commented 1 year ago

@wwoodgate Any chance you could share some extra details about what issues you've run into with the Planetary Computer API keys? I might have a theory about what's going wrong, and want to test it out.

wwoodgate commented 1 year ago

Hey @robbibt, I always got stuck with the "HTTPError: 403 Client Error: Forbidden for url: ..." when searching the item collection, even after adding a custom header with my API key:

Create custom headers including the Authorization header with the API key headers = { 'Authorization': f'Bearer {api_key}', } Initialize the STAC client client = Client.open(stac_api_url, headers=headers)

But I've never done anything like this before so didn't have a working example to go by.

robbibt commented 1 year ago

Hey @wwoodgate, I've been able to load data from Sentinel-1 on the Sandbox - here's the steps I followed:

  1. Register for a Planetary Computer account, then sign into the Developer Portal: https://planetarycomputer.developer.azure-api.net/ (more info here)
  2. Under "Subscriptions", you should see "Primary key" - copy the primary API key (this page took a while to load for me): image
  3. On the DEA Sandbox, launch a new terminal: image
  4. Type planetarycomputer configure. When prompted, paste in your copied API key: image
  5. Now you should be able to load Sentinel-1 data via the Sandbox. For example:
    
    import pystac_client
    import planetary_computer
    import odc.stac

from dea_tools.dask import create_local_dask_cluster client = create_local_dask_cluster(return_client=True)

Open a client pointing to the Microsoft Planetary Computer data catalogue

catalog = pystac_client.Client.open( "https://planetarycomputer.microsoft.com/api/stac/v1", modifier=planetary_computer.sign_inplace, )

Search for data

bbox = (153.110, -26.841, 153.153, -26.803)
search = catalog.search( collections=["sentinel-1-rtc"], bbox=bbox, datetime="2023-08-01/2023-09-09" ) items = search.item_collection() print(f"Found {len(items)} items")

Lazily load Sentinel-1 data with ODC STAC

ds = odc.stac.load( items, bbox=bbox, bands=["vv", "vh"], crs="utm", resolution=10, groupby="solar_day", chunks={}, )

Load into memory with Dask

ds.load()

Plot

ds.isel(time=0).vv.plot(robust=True)


![image](https://github.com/GeoscienceAustralia/dea-notebooks/assets/17680388/e40c5786-786b-4c98-9a2a-6752064319ef)

If you run into any issues, I'd try restarting your notebook kernel after you configure your API, then trying again!
wwoodgate commented 1 year ago

Nailed it @robbibt ! This works perfectly. I'll teach this in tomorrow's prac :)

No issues and didn't need to restart the notebook, very smooth.

Many thanks!! Will