yannforget / landsatxplore

Search and download Landsat scenes from EarthExplorer.
MIT License
223 stars 98 forks source link

Downloading Sentinel data #27

Open yannforget opened 3 years ago

yannforget commented 3 years ago

Hello, Thanks for the great tool! A follow-up question for this issue: Where exactly do these hashes (?) for building the download URL come from?

DATASETS = {
    "LANDSAT_TM_C1": "5e83d08fd9932768",
    "LANDSAT_ETM_C1": "5e83a507d6aaa3db",
    "LANDSAT_8_C1": "5e83d0b84df8d8c2"

I am currently trying to also download Sentinel-2 data using landsatxplore, because downloading it from ESA's scihub (and sentinelsat etc.) is now harder as old data moved into a longterm archive. Searching for S2-Data already works by using the "Sentinel_2A" dataset name. However, I cannot figure out the constant for S2-Data. Thanks, Guido

Originally posted by @griembauer in https://github.com/yannforget/landsatxplore/issues/12#issuecomment-747435091

yannforget commented 3 years ago

Hi @griembauer,

I found the hashes using the developer tools in Firefox (Network tab). It's in a POST request before the actual download. It seems to be 5e83a42c6eba8084 for both S2A and S2B. However there is another value in the URL that is used to identify the scene and I don't know how to get it: for instance https://earthexplorer.usgs.gov/download/5e83a42c6eba8084/11715615/EE/ for the scene L1C_T14TQN_A028609_20201213T172724

Sorry but I don't think I'll be able to go more in depth before the next month!

Yann

griembauer commented 3 years ago

Great, thanks for your quick reply! 11715615 appears to be the entity_Id of this Sentinel-2 scene, e.g.:

>>> scenes = api.search(dataset='SENTINEL_2A',latitude=19.53,longitude=-1.53,start_date='2020-10-01',end_date='2020-12-01',max_cloud_cover=30)
>>> for scene in scenes: print(scene['entityId'])
... 
11029040
11170324
11203052
11244932
11352455
11309345
11397978
11417457
11447470
11467141
11503489

--> From a first look it appears that apart from the hash only util.py/guess_dataset (because of the ID convention), earthexplorer.py/_download (because the file is in .zip, not .tar.gz as for Landsat?) and the cli.py would small adaptions for landsatxplore to also support sentinel-2. I will also see if I can get into it soon! Best, Guido

yannforget commented 3 years ago

Thanks!

Yes, I think you are right. A rapid hack would be: