Unable to download TROPOMI data files locally

[jrbourbeau]

I ran into someone trying to download some TROPOMI L2 data locally to their laptop and ran into an error. Here's a similar reproducer:

import earthaccess
earthaccess.login()

results = earthaccess.search_data(short_name="S5P_L2__CH4____HiR", count=2)
earthaccess.download(results, "./test")

which gives the following error

Granules found: 37496
 Getting 2 granules, approx download size: 0.09 GB
QUEUEING TASKS | : 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 765.38it/s]
PROCESSING TASKS | :   0%|                                                                                                                                                                                                                                           | 0/2 [00:00<?, ?it/s]Error while downloading the file S5P_RPRO_L2__CH4____20180430T034250_20180430T052420_02820_03_020400_20221107T155112.nc
Traceback (most recent call last):
  File "/Users/james/projects/nsidc/earthaccess/earthaccess/store.py", line 564, in _download_file
    r.raise_for_status()
  File "/Users/james/mambaforge/envs/nasa/lib/python3.9/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://data.gesdisc.earthdata.nasa.gov/data/S5P_TROPOMI_Level2/S5P_L2__CH4____HiR.2/2018/120/S5P_RPRO_L2__CH4____20180430T034250_20180430T052420_02820_03_020400_20221107T155112.nc

@betolink any idea what might be happening here?

[betolink]

@jrbourbeau I think this might be related to the EULA from GES_DISC according to the dataset record this data requires the users to accept the Sentinel EULA. Unfortunately there is not a way to accept the EULAs programmatically. NASA was working on improving the error messages to let us know that this was forbidden because the EULA has not been accepted.

Solution: the user needs to visit the NASA EDL page and manually accept the Sentinel EULA and while there accepting the other EULAs won't hurt.

[jrbourbeau]

Ah, I see. Thank you for clarifying @betolink. I accepted the Sentinel and a couple of other GES DISC EULAs and am able to download now 🚀

[MattF-NSIDC]

@betolink Can we detect this "EULA not accepted" case and provide a better message to users? Maybe even a link to visit to do the acceptance flow?

[betolink]

I think it should be possible, I'm not sure if this is per DAAC or across all CMR but maybe we should revisit #36 and at least pass the error to the user if it's related. @MattF-NSIDC

[MattF-NSIDC]

Wonderful, should have known you'd already thought of this :)

I made a quick edit to that issue's description. EDIT: And updated the title and pinned it :)