search

nsidc / earthaccess

Python Library for NASA Earthdata APIs
https://earthaccess.readthedocs.io/
MIT License
394 stars 79 forks source link

get and open should also work with plain links #43

Closed betolink closed 2 years ago

betolink commented 2 years ago

right now, get and open require a DataGranule array, if a user already has a list of files from a NASA DAAC, get and open should work the same way.

andypbarrett commented 2 years ago

Do you have ideas on how to do this? I have just this use case right now.

betolink commented 2 years ago

I think we just need to add handlers for get and open to check if we're dealing with a DataGranule list or simple links, but I was waiting to have the fsspec feature first. All the open and get operations should be done with fsspec and that requires a heavy refactoring of the EDL authentication. I hope to work on this next week.

betolink commented 2 years ago

There is a WIP, we can download granules by passing only a list of files to the GET method but they links have to be HTTPS. We still need to implement the logic to infer the data center and credentials to use if the links are in S3.

if we have a list that looks like this we can now use the Store:

granules = ['https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/ASTGTM.003/ASTGTMV003_N57W103_dem.tif',
     'https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/ASTGTM.003/ASTGTMV003_N59W109_dem.tif',
     'https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/ASTGTM.003/ASTGTMV003_N58W104_dem.tif',
     'https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/ASTGTM.003/ASTGTMV003_N55W109_dem.tif',
     'https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/ASTGTM.003/ASTGTMV003_N54W103_dem.tif']

store = Store(auth)
files = store.get(granules, local_path="./data/")