Applied-GeoSolutions / gips

Geospatial Image Processing System
GNU General Public License v3.0
17 stars 5 forks source link

Sentinel2 esa fetch fail #507

Closed bhbraswell closed 4 years ago

bhbraswell commented 4 years ago

It's been a while since I have tried to work with S2 so I could be missing something. But I'm getting a IOError: Expected wget exit status 0, got 8 on a pretty reasonable request. See full spewage below. Is this a thing?

root@c170adafc580:/gips# gips_process sentinel2 -p ndvi -d 2019-05-01,2019-05-31 -s /archive/vector/boundary.shp -v5 --fetch
GIPS Data Processing (v0.0.0-dev)
Retrieving inventory for site boundary-0 for date range 2019-05-01 - 2019-05-31 (days 1-366)
Computed date range for processing: 2019-05-01 - 2019-05-31
Computed date range for processing: 2019-05-01 - 2019-05-31
Computed date range for processing: 2019-05-01 - 2019-05-31
queried ATD L1C 55HBA 2019-05-01 00:00:00, found S2A_MSIL1C_20190501T002101_N0207_R116_T55HBA_20190501T015632.zip
Fetching S2A_MSIL1C_20190501T002101_N0207_R116_T55HBA_20190501T015632.zip
Authentication selected: Basic realm="OData service"

     0K                                      0%  443K=0.7shttps://scihub.copernicus.eu/dhus/odata/v1/Products('c4e1ac40-1fee-4eab-82e2-024d39c9278d')/$value:
2019-07-29 20:32:35 ERROR 403: Forbidden.
Fatal: 1 error(s) occurred:
Error performing asset download '(https://scihub.copernicus.eu/dhus/odata/v1/Products('c4e1ac40-1fee-4eab-82e2-024d39c9278d')/$value)':
Traceback (most recent call last):
  File "/gips/gips/utils.py", line 619, in cli_error_handler
    yield
  File "/gips/gips/data/sentinel2/sentinel2.py", line 516, in download_esa
    p.returncode))
IOError: Expected wget exit status 0, got 8

tempdir: infintesimal delay on deletion
ircwaves commented 4 years ago

I've mostly been using Google Cloud Platform Storage for Sentinel-2. I'll try out an ESA fetch.

ircwaves commented 4 years ago

I didn't get that error, but it is slower than .... molasses:

icooke@north:~$ gips_inventory sentinel2 -t 55HBA -d 2019-5-1 --fetch -v 5 
GIPS Data Inventory (v0.15.0)
Retrieving inventory for site tiles for date range 2019-05-01 - 2019-05-01 (days 1-366)
Computed date range for processing: 2019-05-01 - 2019-05-01
Computed date range for processing: 2019-05-01 - 2019-05-01
queried ATD L1C 55HBA 2019-05-01 00:00:00, found S2A_MSIL1C_20190501T002101_N0207_R116_T55HBA_20190501T015632.zip
Fetching S2A_MSIL1C_20190501T002101_N0207_R116_T55HBA_20190501T015632.zip
Authentication selected: Basic realm="OData service"

     0K                                      0%  246K=0.2s
     0K
     0K                                      0%  423K=0.1s
     0K
     0K                                      0%  766K=0.7s
     0K
     0K                                      0%  591K=0.5s
     0K
     0K ........ ..                          1% 1.43M=6.7s
     0K ,,,,,,,, ,,...... ......             2% 1.62M=7.3s
bhbraswell commented 4 years ago

Haha thanks I'll check my creds but also, I'll just switch to "gs". Don't know what I was thinking OK...... Thanks

ircwaves commented 4 years ago

Well, it's good to know that ESA fetch is basically not usable at this point.