Unable to access multiple observatories using individual names

smithara commented 4 years ago

I am trying to access two observatories at once, i.e. like:

request = SwarmRequest(server_url)
request.set_collection("SW_OPER_AUX_OBSM2_:ABG", "SW_OPER_AUX_OBSM2_:ABK", )
...

which gives

...
<wps:Input>
    <ows:Identifier>collection_ids</ows:Identifier>
    <wps:Data>
        <wps:ComplexData mimeType="application/json"><![CDATA[{"NSC": ["SW_OPER_AUX_OBSM2_:ABG", "SW_OPER_AUX_OBSM2_:ABK"]}]]></wps:ComplexData>
    </wps:Data>
</wps:Input>
...

Should this be possible? It seems to return data containing only the first observatory. Is the usage of the "NSC" tag there correct?

I am trying to understand the implementation better but getting a bit lost!

```python #### Additional config for viresclient #### IAGA_codes = ['AAA', 'AAE', 'ABG', 'ABK', 'ABN', 'AGN', 'AIA', 'ALE', 'ALH', 'ALM', 'AML', 'AMS', 'AMT', 'AMU', 'ANN', 'API', 'AQU', 'ARC', 'ARE', 'ARK', 'ARS', 'ASC', 'ASH', 'ASO', 'ASP', 'BAG', 'BDE', 'BDV', 'BEL', 'BFE', 'BFO', 'BGY', 'BJI', 'BJN', 'BLC', 'BLT', 'BMT', 'BNG', 'BOU', 'BOX', 'BRD', 'BRT', 'BRW', 'BSL', 'BYR', 'CAO', 'CAX', 'CBB', 'CBI', 'CCS', 'CDP', 'CKI', 'CLF', 'CLH', 'CMO', 'CNB', 'CNH', 'COI', 'CPA', 'CPL', 'CRP', 'CSY', 'CTA', 'CTO', 'CTS', 'CTX', 'CWE', 'CYG', 'CZT', 'DAL', 'DAV', 'DBN', 'DED', 'DIK', 'DLR', 'DLT', 'DMC', 'DOB', 'DOU', 'DRV', 'DUR', 'DVS', 'EBR', 'EGS', 'EIC', 'EKT', 'ELT', 'ESA', 'ESK', 'ETT', 'EYR', 'FAN', 'FCC', 'FRA', 'FRD', 'FRN', 'FSP', 'FTN', 'FUQ', 'FUR', 'GAN', 'GCK', 'GDH', 'GLM', 'GLN', 'GNA', 'GNG', 'GRM', 'GUA', 'GUI', 'GUL', 'GWC', 'GZH', 'HAD', 'HBA', 'HBK', 'HER', 'HIS', 'HLP', 'HLW', 'HNA', 'HON', 'HRB', 'HRN', 'HTY', 'HUA', 'HVN', 'HYB', 'IBD', 'IPM', 'IQA', 'IRT', 'ISK', 'IZN', 'JAI', 'JCO', 'JRV', 'KAK', 'KDU', 'KEP', 'KGD', 'KHB', 'KIR', 'KIV', 'KLI', 'KMH', 'KNY', 'KNZ', 'KOD', 'KOR', 'KOU', 'KPG', 'KRC', 'KSH', 'KZN', 'LAA', 'LDV', 'LEN', 'LER', 'LGR', 'LIV', 'LMM', 'LNN', 'LNP', 'LON', 'LOV', 'LPB', 'LQA', 'LRM', 'LRV', 'LUA', 'LVV', 'LWI', 'LYC', 'LZH', 'MAB', 'MAW', 'MBC', 'MBO', 'MCP', 'MCQ', 'MEA', 'MFP', 'MGD', 'MID', 'MIR', 'MIZ', 'MLT', 'MMB', 'MMK', 'MNK', 'MOL', 'MOS', 'MRN', 'MUB', 'MUT', 'MZL', 'NAI', 'NAQ', 'NCK', 'NEW', 'NGK', 'NGP', 'NKK', 'NMP', 'NRD', 'NUR', 'NVL', 'NVS', 'NWS', 'OAS', 'ODE', 'OTT', 'PAB', 'PAF', 'PAG', 'PBQ', 'PCU', 'PEG', 'PET', 'PHU', 'PIL', 'PIO', 'PLR', 'PMG', 'PND', 'POD', 'POT', 'PPT', 'PRU', 'PSM', 'PST', 'PTU', 'QGZ', 'QIX', 'QSB', 'QUE', 'QZH', 'RBD', 'RES', 'RKT', 'ROB', 'RSV', 'SAB', 'SBA', 'SBL', 'SCO', 'SED', 'SFS', 'SGE', 'SHE', 'SHL', 'SHU', 'SIL', 'SIT', 'SJG', 'SKT', 'SMG', 'SNA', 'SOD', 'SPA', 'SPG', 'SPT', 'SSH', 'SSO', 'STJ', 'STO', 'SUA', 'SVD', 'SWI', 'SZT', 'TAH', 'TAL', 'TAM', 'TAN', 'TDC', 'TEH', 'TEN', 'TEO', 'TFS', 'THJ', 'THL', 'THY', 'TIK', 'TIR', 'TKH', 'TKT', 'TMK', 'TND', 'TNG', 'TOK', 'TOL', 'TOO', 'TRD', 'TRO', 'TRW', 'TSU', 'TTB', 'TUC', 'TUN', 'UBA', 'UJJ', 'UPS', 'VAL', 'VIC', 'VLA', 'VLJ', 'VNA', 'VOS', 'VQS', 'VSK', 'VSS', 'WAT', 'WHN', 'WHS', 'WIA', 'WIC', 'WIK', 'WIL', 'WIT', 'WNG', 'YAK', 'YKC', 'YSS'] # Collection names obs_collections = { "AUX_OBSM": ["SW_OPER_AUX_OBSM2_"], "AUX_OBSS": ["SW_OPER_AUX_OBSS2_"] } # Add "SW_OPER_AUX_OBSM2_:ABG" etc. obs_collections["AUX_OBSM"].extend(f"SW_OPER_AUX_OBSM2_:{code}" for code in IAGA_Codes) # Add "SW_OPER_AUX_OBSS2_:ABG" etc. obs_collections["AUX_OBSS"].extend(f"SW_OPER_AUX_OBSS2_:{code}" for code in IAGA_Codes) # Measurement variable names obs_measurements = { "AUX_OBSM": ["B_NEC", "F", "IAGA_code", "Quality"], "AUX_OBSS": ["B_NEC", "F", "IAGA_code", "Quality"], } sampling = { "AUX_OBSM": "PT60S", "AUX_OBSS": "PT1S" } from viresclient import SwarmRequest SwarmRequest.COLLECTIONS.update(obs_collections) SwarmRequest.PRODUCT_VARIABLES.update(obs_measurements) SwarmRequest.COLLECTION_SAMPLING_STEPS.update(sampling) #### Request attempt #### server_url = 'https://staging.viresdisc.vires.services/ows' request = SwarmRequest(server_url) request.set_collection("SW_OPER_AUX_OBSM2_:ABG", "SW_OPER_AUX_OBSM2_:ABK", ) request.set_products( measurements=['B_NEC', 'F', 'IAGA_code', 'Quality'], auxiliaries=['QDLat', 'QDLon', 'MLT'], ) data = request.get_between( start_time='2016-01-01T10:00:00Z', end_time='2016-01-01T14:00:00Z', asynchronous=False, ) ds = data.as_xarray() ds ```

pacesm commented 4 years ago

Should this be possible? It seems to return data containing only the first observatory.

It is possible to request collections like this

{
  "NSC": ["SW_OPER_AUX_OBSM2_:ABG", "SW_OPER_AUX_OBSM2_:ABK"]
}

but the result will contain data from the first collection only. The reason is that the two collections are of the same type and thus they contain the same variables. The requested variables are read from the first collection while there is no variable left to be read from the second.

To get data from two (or more) sensors in one shot make a request like this:

{
  "ABG": ["SW_OPER_AUX_OBSM2_:ABG"],
  "ABK": ["SW_OPER_AUX_OBSM2_:ABK"]
}

Note that when we request data from different Swarm spacecraft we make a request like

{
  "Alpha":["SW_OPER_MAGA_LR_1B"],
  "Bravo":["SW_OPER_MAGB_LR_1B"],
  "Charlie":["SW_OPER_MAGC_LR_1B"]
}

Is the usage of the "NSC" tag there correct?

This label can be anything. The server makes no use of it. It is used only by web GUI requests.

smithara commented 4 years ago

Thank you for the explanation!

I made the client fix for this along with the initial AUX_OBS configuration https://github.com/ESA-VirES/VirES-Python-Client/commit/17493bb4de64fdc10e7454f88b0f21e700179bc2

Though I should really add the option to make independent requests automatically and merge the results using some user-selectable logic

ESA-VirES / VirES-Server

Unable to access multiple observatories using individual names #157