geodesymiami / rsmas_insar

RSMAS InSAR code
https://rsmas-insar.readthedocs.io/
GNU General Public License v3.0
58 stars 22 forks source link

Identify download errors in `minsarApp.bash` by searching for `HTTP Error 503` #540

Open falkamelung opened 4 months ago

falkamelung commented 4 months ago

When download using ssara from the ASF fails, we commonly get the error below. We need to check the *.e file (or similar) for this string and display a proper error message (same when the Unavco ssara server is down). Here the message:

tail ssara_listing.txt
Running SSARA API Query:  https://web-services.unavco.org/brokered/ssara/api/sar/search?platform=SENTINEL-1A%2CSENTINEL-1B&relativeOrbit=128&start=2023-10-01&processingLevel=L0%2CL1.0%2CSLC&maxResults=20000&intersectsWith=Polygon%28%28-93.00+-1.00%2C+-93.00+0.15%2C+-89.00+0.15%2C+-89.00+-1.00%2C+-93.00+-1.00%29%29&asfResponseTimeout=300
//login3/scratch/05861/tg851601/GalapagosSenDT128/SLC[1014] tail -50 ssara_listing.txt
Running SSARA API Query:  https://web-services.unavco.org/brokered/ssara/api/sar/search?platform=SENTINEL-1A%2CSENTINEL-1B&relativeOrbit=128&start=2023-10-01&processingLevel=L0%2CL1.0%2CSLC&maxResults=20000&intersectsWith=Polygon%28%28-93.00+-1.00%2C+-93.00+0.15%2C+-89.00+0.15%2C+-89.00+-1.00%2C+-93.00+-1.00%29%29&asfResponseTimeout=300
//login3/scratch/05861/tg851601/GalapagosSenDT128/SLC[1015] ssara_federated_query.py  --relativeOrbit=128 --intersectsWith='Polygon((-93.00 -1.00, -93.00 0.15, -89.00 0.15, -89.00 -1.00, -93.00 -1.00))' --platform=SENTINEL-1A,SENTINEL-1B --start=2023-10-01 --parallel=5 --maxResults=20000 --print
Running SSARA API Query:  https://web-services.unavco.org/brokered/ssara/api/sar/search?platform=SENTINEL-1A%2CSENTINEL-1B&relativeOrbit=128&start=2023-10-01&processingLevel=L0%2CL1.0%2CSLC&maxResults=20000&intersectsWith=Polygon%28%28-93.00+-1.00%2C+-93.00+0.15%2C+-89.00+0.15%2C+-89.00+-1.00%2C+-93.00+-1.00%29%29
Traceback (most recent call last):
  File "/work2/05861/tg851601/stampede2/code/rsmas_insar/tools/SSARA/ssara_federated_query.py", line 425, in <module>
    main(sys.argv[1:])
  File "/work2/05861/tg851601/stampede2/code/rsmas_insar/tools/SSARA/ssara_federated_query.py", line 178, in main
    f = urlopen(ssara_url)
        ^^^^^^^^^^^^^^^^^^
  File "/work2/05861/tg851601/stampede2/code/rsmas_insar/tools/miniconda3/lib/python3.11/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work2/05861/tg851601/stampede2/code/rsmas_insar/tools/miniconda3/lib/python3.11/urllib/request.py", line 525, in open
    response = meth(req, response)
               ^^^^^^^^^^^^^^^^^^^
  File "/work2/05861/tg851601/stampede2/code/rsmas_insar/tools/miniconda3/lib/python3.11/urllib/request.py", line 634, in http_response
    response = self.parent.error(
               ^^^^^^^^^^^^^^^^^^
  File "/work2/05861/tg851601/stampede2/code/rsmas_insar/tools/miniconda3/lib/python3.11/urllib/request.py", line 563, in error
    return self._call_chain(*args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/work2/05861/tg851601/stampede2/code/rsmas_insar/tools/miniconda3/lib/python3.11/urllib/request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/work2/05861/tg851601/stampede2/code/rsmas_insar/tools/miniconda3/lib/python3.11/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 503: Service Temporarily Unavailable
//login3/scratch/05861/tg851601/GalapagosSenDT128/SLC[1016]