search

geodesymiami / rsmas_insar

RSMAS InSAR code
https://rsmas-insar.readthedocs.io/
GNU General Public License v3.0
62 stars 23 forks source link

check_services.py for ASF #507

Closed falkamelung closed 3 years ago

falkamelung commented 3 years ago

Hi Josh, Since yesterday night the ASF data download is down. This would ow is an excellent time to get check_services.py to work — if you can spare some time. I am getting the output below (also with debugger switched on):

check_services.py --all
demServer is                    ONLINE (fast)
downloadASF list service is     ONLINE (fast)
downloadASF download service is ONLINE     
jetstream server is             ONLINE (fast)
insarmaps server is             ONLINE (fast)
$WORK is                        ONLINE (fast)
[3] > /work2/05861/tg851601/stampede2/code/rsmas_insar/minsar/check_services.py(19)check_server_status()
-> command = "wget --spider {} --connect-timeout={} --tries=3 {} ".format(server_url, timeout, option_str)
(Pdb++) n
[3] > /work2/05861/tg851601/stampede2/code/rsmas_insar/minsar/check_services.py(20)check_server_status()
-> process = subprocess.run(command.split(), capture_output=True)
(Pdb++) n
[3] > /work2/05861/tg851601/stampede2/code/rsmas_insar/minsar/check_services.py(21)check_server_status()
-> output = str(process.stderr)
(Pdb++) output
*** NameError: name 'output' is not defined
(Pdb++) process
CompletedProcess(args=['wget', '--spider', 'https://datapool.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip', '--connect-timeout=0.1', '--tries=3'], returncode=4, stdout=b'', stderr=b'Spider mode enabled. Check if remote file exists.\n--2021-10-16 11:27:06--  https://datapool.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip\nResolving datapool.asf.alaska.edu (datapool.asf.alaska.edu)... 137.229.86.206\nConnecting to datapool.asf.alaska.edu (datapool.asf.alaska.edu)|137.229.86.206|:443... failed: Connection timed out.\nRetrying.\n\nSpider mode enabled. Check if remote file exists.\n--2021-10-16 11:27:07--  (try: 2)  https://datapool.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip\nConnecting to datapool.asf.alaska.edu (datapool.asf.alaska.edu)|137.229.86.206|:443... failed: Connection timed out.\nRetrying.\n\nSpider mode enabled. Check if remote file exists.\n--2021-10-16 11:27:09--  (try: 3)  https://datapool.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip\nConnecting to datapool.asf.alaska.edu (datapool.asf.alaska.edu)|137.229.86.206|:443... failed: Connection timed out.\nGiving up.\n\n')
(Pdb++) process.stderr
b'Spider mode enabled. Check if remote file exists.\n--2021-10-16 11:27:06--  https://datapool.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip\nResolving datapool.asf.alaska.edu (datapool.asf.alaska.edu)... 137.229.86.206\nConnecting to datapool.asf.alaska.edu (datapool.asf.alaska.edu)|137.229.86.206|:443... failed: Connection timed out.\nRetrying.\n\nSpider mode enabled. Check if remote file exists.\n--2021-10-16 11:27:07--  (try: 2)  https://datapool.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip\nConnecting to datapool.asf.alaska.edu (datapool.asf.alaska.edu)|137.229.86.206|:443... failed: Connection timed out.\nRetrying.\n\nSpider mode enabled. Check if remote file exists.\n--2021-10-16 11:27:09--  (try: 3)  https://datapool.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip\nConnecting to datapool.asf.alaska.edu (datapool.asf.alaska.edu)|137.229.86.206|:443... failed: Connection timed out.\nGiving up.\n\n'

Another try:

(Pdb++) process
CompletedProcess(args=['wget', '--spider', 'https://datapool.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip', '--connect-timeout=0.1', '--tries=3'], returncode=8, stdout=b'', stderr=b'Spider mode enabled. Check if remote file exists.\n--2021-10-16 11:53:35--  https://datapool.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip\nResolving datapool.asf.alaska.edu (datapool.asf.alaska.edu)... 137.229.86.206\nConnecting to datapool.asf.alaska.edu (datapool.asf.alaska.edu)|137.229.86.206|:443... connected.\nHTTP request sent, awaiting response... 307 TEMPORARY REDIRECT\nLocation: https://sentinel1.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip [following]\nSpider mode enabled. Check if remote file exists.\n--2021-10-16 11:53:35--  https://sentinel1.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip\nResolving sentinel1.asf.alaska.edu (sentinel1.asf.alaska.edu)... 143.204.166.62, 143.204.166.12, 143.204.166.95, ...\nConnecting to sentinel1.asf.alaska.edu (sentinel1.asf.alaska.edu)|143.204.166.62|:443... connected.\nHTTP request sent, awaiting response... 500 Internal Server Error\n--2021-10-16 11:53:37--  (try: 2)  https://sentinel1.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip\nConnecting to sentinel1.asf.alaska.edu (sentinel1.asf.alaska.edu)|143.204.166.62|:443... connected.\nHTTP request sent, awaiting response... 302 Moved Temporarily\nLocation: https://urs.earthdata.nasa.gov/oauth/authorize?client_id=BO_n7nTIlMljdvU6kRRB3g&response_type=code&redirect_uri=https://sentinel1.asf.alaska.edu/login&state=%2FSLC%2FSA%2FS1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip&app_type=401 [following]\nSpider mode enabled. Check if remote file exists.\n--2021-10-16 11:53:38--  https://urs.earthdata.nasa.gov/oauth/authorize?client_id=BO_n7nTIlMljdvU6kRRB3g&response_type=code&redirect_uri=https://sentinel1.asf.alaska.edu/login&state=%2FSLC%2FSA%2FS1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip&app_type=401\nResolving urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)... 198.118.243.33, 2001:4d0:241a:4081::89\nConnecting to urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)|198.118.243.33|:443... connected.\nHTTP request sent, awaiting response... 401 Unauthorized\nConnecting to urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)|198.118.243.33|:443... connected.\nHTTP request sent, awaiting response... 302 Found\nLocation: https://sentinel1.asf.alaska.edu/login?code=cb801522e0c4624be16c959db8fe514cc506845010c32aa718aac67bf524f411&state=%2FSLC%2FSA%2FS1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip [following]\nSpider mode enabled. Check if remote file exists.\n--2021-10-16 11:53:38--  https://sentinel1.asf.alaska.edu/login?code=cb801522e0c4624be16c959db8fe514cc506845010c32aa718aac67bf524f411&state=%2FSLC%2FSA%2FS1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip\nConnecting to sentinel1.asf.alaska.edu (sentinel1.asf.alaska.edu)|143.204.166.62|:443... connected.\nHTTP request sent, awaiting response... 404 Not Found\nRemote file does not exist -- broken link!!!\n\n')
(Pdb++) process.stderr
b'Spider mode enabled. Check if remote file exists.\n--2021-10-16 11:53:35--  https://datapool.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip\nResolving datapool.asf.alaska.edu (datapool.asf.alaska.edu)... 137.229.86.206\nConnecting to datapool.asf.alaska.edu (datapool.asf.alaska.edu)|137.229.86.206|:443... connected.\nHTTP request sent, awaiting response... 307 TEMPORARY REDIRECT\nLocation: https://sentinel1.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip [following]\nSpider mode enabled. Check if remote file exists.\n--2021-10-16 11:53:35--  https://sentinel1.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip\nResolving sentinel1.asf.alaska.edu (sentinel1.asf.alaska.edu)... 143.204.166.62, 143.204.166.12, 143.204.166.95, ...\nConnecting to sentinel1.asf.alaska.edu (sentinel1.asf.alaska.edu)|143.204.166.62|:443... connected.\nHTTP request sent, awaiting response... 500 Internal Server Error\n--2021-10-16 11:53:37--  (try: 2)  https://sentinel1.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip\nConnecting to sentinel1.asf.alaska.edu (sentinel1.asf.alaska.edu)|143.204.166.62|:443... connected.\nHTTP request sent, awaiting response... 302 Moved Temporarily\nLocation: https://urs.earthdata.nasa.gov/oauth/authorize?client_id=BO_n7nTIlMljdvU6kRRB3g&response_type=code&redirect_uri=https://sentinel1.asf.alaska.edu/login&state=%2FSLC%2FSA%2FS1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip&app_type=401 [following]\nSpider mode enabled. Check if remote file exists.\n--2021-10-16 11:53:38--  https://urs.earthdata.nasa.gov/oauth/authorize?client_id=BO_n7nTIlMljdvU6kRRB3g&response_type=code&redirect_uri=https://sentinel1.asf.alaska.edu/login&state=%2FSLC%2FSA%2FS1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip&app_type=401\nResolving urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)... 198.118.243.33, 2001:4d0:241a:4081::89\nConnecting to urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)|198.118.243.33|:443... connected.\nHTTP request sent, awaiting response... 401 Unauthorized\nConnecting to urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)|198.118.243.33|:443... connected.\nHTTP request sent, awaiting response... 302 Found\nLocation: https://sentinel1.asf.alaska.edu/login?code=cb801522e0c4624be16c959db8fe514cc506845010c32aa718aac67bf524f411&state=%2FSLC%2FSA%2FS1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip [following]\nSpider mode enabled. Check if remote file exists.\n--2021-10-16 11:53:38--  https://sentinel1.asf.alaska.edu/login?code=cb801522e0c4624be16c959db8fe514cc506845010c32aa718aac67bf524f411&state=%2FSLC%2FSA%2FS1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip\nConnecting to sentinel1.asf.alaska.edu (sentinel1.asf.alaska.edu)|143.204.166.62|:443... connected.\nHTTP request sent, awaiting response... 404 Not Found\nRemote file does not exist -- broken link!!!\n\n'
(Pdb++) output
"b'Spider mode enabled. Check if remote file exists.\\n--2021-10-16 11:53:35--  https://datapool.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip\\nResolving datapool.asf.alaska.edu (datapool.asf.alaska.edu)... 137.229.86.206\\nConnecting to datapool.asf.alaska.edu (datapool.asf.alaska.edu)|137.229.86.206|:443... connected.\\nHTTP request sent, awaiting response... 307 TEMPORARY REDIRECT\\nLocation: https://sentinel1.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip [following]\\nSpider mode enabled. Check if remote file exists.\\n--2021-10-16 11:53:35--  https://sentinel1.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip\\nResolving sentinel1.asf.alaska.edu (sentinel1.asf.alaska.edu)... 143.204.166.62, 143.204.166.12, 143.204.166.95, ...\\nConnecting to sentinel1.asf.alaska.edu (sentinel1.asf.alaska.edu)|143.204.166.62|:443... connected.\\nHTTP request sent, awaiting response... 500 Internal Server Error\\n--2021-10-16 11:53:37--  (try: 2)  https://sentinel1.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip\\nConnecting to sentinel1.asf.alaska.edu (sentinel1.asf.alaska.edu)|143.204.166.62|:443... connected.\\nHTTP request sent, awaiting response... 302 Moved Temporarily\\nLocation: https://urs.earthdata.nasa.gov/oauth/authorize?client_id=BO_n7nTIlMljdvU6kRRB3g&response_type=code&redirect_uri=https://sentinel1.asf.alaska.edu/login&state=%2FSLC%2FSA%2FS1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip&app_type=401 [following]\\nSpider mode enabled. Check if remote file exists.\\n--2021-10-16 11:53:38--  https://urs.earthdata.nasa.gov/oauth/authorize?client_id=BO_n7nTIlMljdvU6kRRB3g&response_type=code&redirect_uri=https://sentinel1.asf.alaska.edu/login&state=%2FSLC%2FSA%2FS1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip&app_type=401\\nResolving urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)... 198.118.243.33, 2001:4d0:241a:4081::89\\nConnecting to urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)|198.118.243.33|:443... connected.\\nHTTP request sent, awaiting response... 401 Unauthorized\\nConnecting to urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)|198.118.243.33|:443... connected.\\nHTTP request sent, awaiting response... 302 Found\\nLocation: https://sentinel1.asf.alaska.edu/login?code=cb801522e0c4624be16c959db8fe514cc506845010c32aa718aac67bf524f411&state=%2FSLC%2FSA%2FS1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip [following]\\nSpider mode enabled. Check if remote file exists.\\n--2021-10-16 11:53:38--  https://sentinel1.asf.alaska.edu/login?code=cb801522e0c4624be16c959db8fe514cc506845010c32aa718aac67bf524f411&state=%2FSLC%2FSA%2FS1A_IW_SLC__1SSV_20160605T114943_20160605T115018_011575_011AEF_98EA.zip\\nConnecting to sentinel1.asf.alaska.edu (sentinel1.asf.alaska.edu)|143.204.166.62|:443... connected.\\nHTTP request sent, awaiting response... 404 Not Found\\nRemote file does not exist -- broken link!!!\\n\\n'"