search

nasa / EMIT-Data-Resources

This repository provides guides, short how-tos, and tutorials to help users access and work with data from the Earth Surface Mineral Dust Source Investigation (EMIT) mission.
Apache License 2.0
132 stars 75 forks source link

s3 access EMIT data #50

Closed kleinlevente closed 5 months ago

kleinlevente commented 8 months ago

Trying to run the s3 access notebook : https://github.com/nasa/EMIT-Data-Resources/blob/main/python/how-tos/How_to_Direct_S3_Access.ipynb but I am running in an access error: ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden

The above exception was the direct cause of the following exception:

PermissionError Traceback (most recent call last) Cell In[9], line 2 1 # Open s3 url ----> 2 fp = fs_s3.open(s3_url, mode='rb') 3 # Open dataset with xarray 4 ds = xr.open_dataset(fp) #Note this only opens the root group (reflectance)

with a final error: PermissionError: Forbidden

I assume this is a setting issue on the COS bucket where access is defined.

amfriesz commented 8 months ago

Hey @kleinlevente,

Are you running the code from a compute instance in AWS us-west-2? Direct S3 access of NASA assets is only allowed if you are working from us-west-2. In all other situations, e.g. working locally, from another AWS region, or from another cloud provider, you would need to use the HTTPS URLS associated with the data assets.