m9brady / eodms-api-client

EODMS API Client for Python
https://m9brady.github.io/eodms-api-client/
MIT License
18 stars 5 forks source link

Could not query multiple beam modes #6

Open ECCCBen opened 2 years ago

ECCCBen commented 2 years ago

Tried to query multiple beam modes in one query but only got one beam mode or all of them.

Tried:

x = EodmsAPI('RCM')
x.query(start='2019-07-01', end='2020-10-20', geometry='Data/CG_EODMS_Query.geojson', polarization='HH HV',
       beam_mode='Low Noise Med Resolution 50m Low Resolution 100m')

Got all beam modes.

And

x = EodmsAPI('RCM')
x.query(start='2019-07-01', end='2020-10-20', geometry='Data/CG_EODMS_Query.geojson', polarization='HH HV',
       beam_mode=['Low Noise, Med Resolution 50m, Low Resolution 100m'])

Only got the first beam mode in the list. Note: if I change the order of the beam modes in the list, I get the first beam mode.

For now, I use all the beam modes and sort the results post query. Thanks!

m9brady commented 2 years ago

Thanks for the report, I'll look into it.

One point of clarification - are you passing the list of beam modes like this (single-item list):

beam_mode=['Low Noise, Med Resolution 50m, Low Resolution 100m']

or like this (multi-item list):

beam_mode=['Low Noise', 'Medium Resolution 50m', 'Low Resolution 100m']
ECCCBen commented 2 years ago

multi-item list, my bad

m9brady commented 2 years ago

The way multi-value queries have been implemented with EODMS has always been a bit confusing for me and I can't quite figure out what's wrong here. The way this client tries to achieve it is:

RCM.SBEAM='Low Noise','Medium Resolution 50m','Low Resolution 100m'

I've also tried based on sneaking a look at NRCan's client and it looks like they craft multi-value query params like so:

(RCM.SBEAM='Low Noise' OR RCM.SBEAM='Medium Resolution 50m' OR RCM.SBEAM='Low Resolution 100m')

but using ☝️ here actually ends up messing with the polarization param as well, returning even more not-asked-for granules.

I also noticed that when I run this query for a single beam mode:

x.query(start='2019-07-01', end='2020-10-20', geometry='Data/CG_EODMS_Query.geojson', polarization='HH HV', beam_mode='Medium Resolution 50m')

I get way too many results and the polarization/beam mode filters don't appear to have any effect... No idea why Low Noise works fine but not Medium Resolution 50m. I will revisit this later when I've got more time to examine.

m9brady commented 2 years ago

Just tried a similar query with the NRCan client and I think the API is not correctly applying RCM.SBEAM filters when the desired beam mode has a number in it:

from eodms_rapi import EODMSRAPI
from netrc import netrc
user, _, pw = netrc().authenticators('data.eodms-sgdot.nrcan-rncan.gc.ca')
rapi = EODMSRAPI(user, pw)

# filter params
dates = [{'start': '20201021_000000', 'end': '20201021_235959'}]
filters = {'Beam Mode Type': ('=', ['Medium Resolution 50m']), 'Polarization': ('=', ['HH HV'])}
feats = [('intersects', 'Data/CG_EODMS_Query.geojson')]

# I expect this to return only Med Res 50 based on the above filters
rapi.search("RCMImageProducts", filters=filters, features=feats, dates=dates)
rapi.get_results('full')

[fff[1] for ff in rapi.results for fff in ff['metadata'] if fff[0] == 'Beam Mode Description']
['ScanSAR 50m Resolution 350km Swath B',
 'ScanSAR 50m Resolution 350km Swath B',
 'ScanSAR Low Noise 100m Resolution 350km Swath B',
 'ScanSAR Low Noise 100m Resolution 350km Swath B',
 'ScanSAR Low Noise 100m Resolution 350km Swath B']

🤷‍♂️