search

asfadmin / Discovery-asf_search

BSD 3-Clause "New" or "Revised" License
131 stars 45 forks source link

[Bug/Question] VH tifs only returned for OPERA-RTC-S1 - but polarizations are all VV in metadata. How do we get h5 products? #231

Closed cmarshak closed 11 months ago

cmarshak commented 12 months ago

I am trying to recreate this search: https://search.asf.alaska.edu/#/?polygon=POINT(-99.8796%2016.8421)&zoom=9.133&center=-99.712,16.370&resultsLoaded=true&dataset=OPERA-S1&granule=OPERA_L2_RTC-S1_T078-165486-IW2_20231024T004829Z_20231024T091416Z_S1A_30_v1.0&productTypes=RTC

resp = asf.geo_search(intersectsWith=Point(-99.8796, 16.8421).wkt,
                      processingLevel='RTC', 
                      platform=asf.constants.SENTINEL1)
len(resp)

Gives 4 results.

resp = asf.geo_search(intersectsWith=Point(-99.8796, 16.8421).wkt,
                      processingLevel='RTC', 
                      platform=asf.constants.SENTINEL1,
                      polarization=['VV', 'VH'])
len(resp)

Gives 8.

But the latter is only giving me VH. For example:

resp[-1].properties

gives

{'beamModeType': None,
 'browse': ['https://datapool.asf.alaska.edu/BROWSE/OPERA-S1/OPERA_L2_RTC-S1_T041-086893-IW1_20231009T123527Z_20231010T052433Z_S1A_30_v1.0_BROWSE.png',
  's3://asf-cumulus-prod-opera-browse/OPERA_L2_RTC-S1/OPERA_L2_RTC-S1_T041-086893-IW1_20231009T123527Z_20231010T052433Z_S1A_30_v1.0/OPERA_L2_RTC-S1_T041-086893-IW1_20231009T123527Z_20231010T052433Z_S1A_30_v1.0_BROWSE.png',
  'https://datapool.asf.alaska.edu/BROWSE/OPERA-S1/OPERA_L2_RTC-S1_T041-086893-IW1_20231009T123527Z_20231010T052433Z_S1A_30_v1.0_BROWSE_low-res.png',
  's3://asf-cumulus-prod-opera-browse/OPERA_L2_RTC-S1/OPERA_L2_RTC-S1_T041-086893-IW1_20231009T123527Z_20231010T052433Z_S1A_30_v1.0/OPERA_L2_RTC-S1_T041-086893-IW1_20231009T123527Z_20231010T052433Z_S1A_30_v1.0_BROWSE_low-res.png',
  'https://datapool.asf.alaska.edu/BROWSE/OPERA-S1/OPERA_L2_RTC-S1_T041-086893-IW1_20231009T123527Z_20231010T052433Z_S1A_30_v1.0_BROWSE_thumbnail.png',
  's3://asf-cumulus-prod-opera-browse/OPERA_L2_RTC-S1/OPERA_L2_RTC-S1_T041-086893-IW1_20231009T123527Z_20231010T052433Z_S1A_30_v1.0/OPERA_L2_RTC-S1_T041-086893-IW1_20231009T123527Z_20231010T052433Z_S1A_30_v1.0_BROWSE_thumbnail.png'],
 'bytes': None,
 'centerLat': None,
 'centerLon': None,
 'faradayRotation': None,
 'fileID': 'OPERA_L2_RTC-S1_T041-086893-IW1_20231009T123527Z_20231010T052433Z_S1A_30_v1.0',
 'flightDirection': 'DESCENDING',
 'groupID': 'S1A_IWDV_0533_0538_050688_041',
 'granuleType': None,
 'insarStackId': None,
 'md5sum': None,
 'offNadirAngle': None,
 'orbit': 50688,
 'pathNumber': 41,
 'platform': 'Sentinel-1A',
 'pointingAngle': None,
 'polarization': 'VV',
 'processingDate': '2023-10-10T05:32:17Z',
 'processingLevel': 'RTC',
 'sceneName': 'OPERA_L2_RTC-S1_T041-086893-IW1_20231009T123527Z_20231010T052433Z_S1A_30_v1.0',
 'sensor': 'C-SAR',
 'startTime': '2023-10-09T12:35:27Z',
 'stopTime': '2023-10-09T12:35:30Z',
 'url': 'https://datapool.asf.alaska.edu/RTC/OPERA-S1/OPERA_L2_RTC-S1_T041-086893-IW1_20231009T123527Z_20231010T052433Z_S1A_30_v1.0_VH.tif',
 'fileName': 'OPERA_L2_RTC-S1_T041-086893-IW1_20231009T123527Z_20231010T052433Z_S1A_30_v1.0_VH.tif',
 'frameNumber': None}

Note in the above the url is for VH, but the polarization is for VV.

Also, how do I get the h5 files in that search?

cmarshak commented 12 months ago

Currently, replacing VV with VH in the url does the trick and a similar substition for h5 will work.

Again, just providing information for someone using your tool likely before it's even ready for the public ;)

SpicyGarlicAlbacoreRoll commented 11 months ago

Thank you for opening an issue! Full opera support is pending our next release (very soon). The polarization/url discrepancy is a quirk of Opera VV/VH products being a single CMR entry. All ASFProducts have the umm response from CMR available, and if you were to inspect the umm of one of your results like so:

response[-1].umm

you would see something like this:

{
...
"GranuleUR": "OPERA_L2_RTC-S1_T078-165486-IW2_20231105T004829Z_20231106T002602Z_S1A_30_v1.0",
"AdditionalAttributes": [
        ...
        {
            "Name": "POLARIZATION",
            "Values": [
                "VV",
                "VH"
            ]
        },
    ],
    "RelatedUrls": [
        {
...
            "URL": "https://datapool.asf.alaska.edu/RTC/OPERA-S1/OPERA_L2_RTC-S1_T078-165486-IW2_20231105T004829Z_20231106T002602Z_S1A_30_v1.0_VH.tif",
            "Type": "GET DATA",
            "Description": "Download OPERA_L2_RTC-S1_T078-165486-IW2_20231105T004829Z_20231106T002602Z_S1A_30_v1.0_VH.tif",
            "Format": "Not provided"
        },
...
        {
            "URL": "https://datapool.asf.alaska.edu/RTC/OPERA-S1/OPERA_L2_RTC-S1_T078-165486-IW2_20231105T004829Z_20231106T002602Z_S1A_30_v1.0_VV.tif",
            "Type": "GET DATA",
            "Description": "Download OPERA_L2_RTC-S1_T078-165486-IW2_20231105T004829Z_20231106T002602Z_S1A_30_v1.0_VV.tif",
            "Format": "Not provided"
        },
    ]
}

Note the VV and VH product urls being available for a single product. asf-search naively grabs the first polarization available from the POLARIZATION additional attribute, and the first download url as well.

In the next release the VV, mask, hd5, geotiff, and xml urls will be available in the product properties dict via additionalUrls, where they can be downloaded individually with an authenticated session like so:

asf.download_url(results[0].properties['additionalUrls'][0], "path/to/download", "filename.ext", session)

If you find that you want to download all files for a product (vv, vh, mask, etc), ASFProduct's built in download() with fileType will work!

results[0].download("path/to/download", session=session, fileType=asf.FileDownloadType.ALL_FILES)
SpicyGarlicAlbacoreRoll commented 11 months ago

With the latest release: https://github.com/asfadmin/Discovery-asf_search/releases/tag/v6.7.0