simpeg / aurora

software for processing natural source electromagnetic data
MIT License
14 stars 2 forks source link

Wildcard support for make mth5 from FDSN #277

Open kkappler opened 1 year ago

kkappler commented 1 year ago

Currently, issue #252 works around wildcards by using data_availability tables built by Laura's ipynb get_MT_numbers.ipynb. Using a single * for channels does not work in a request df, but using *F* does work (in limited testing) for pulling all magnetomter channels.

It should be tested that passing ['F', 'Q'] as a channels list works for widescale mth5 building.

kkappler commented 1 year ago

The wildcards do not work for making dataless mth5 (stage 02 of widescale testing). The issue is in the get_inventory_from_df method of mth5/mth5/clients/fdsn.py. around line 463 is the following code:

returned_chan = cha_inv.networks[0].stations[0].channels[0] returned_sta.channels.append(returned_chan)

which assumes there is only one channel returned from the query.

This could possibly be substituted with a loop for each channel. Another solution would be to reform the dataframe without wildcards before the request.

This might involve:

tmp_inv = client.get_stations(ch_row.start, ch_row.end, network=ch_row.network, station=ch_row.station, level="channel")
channels = tmp_inv[0][0].channels
channel_codes =  [x.code for x in channels]

And then we could, for example melt these channel codes over the dataframe row.

The first option is a fairly quick fix, but adds complexity to an already complex piece of code. The second option will be simpler to debug in future if needed. With the second option is that the request_df may have many rows with wildcards, and some uniqueness-checking should be done after it is wildcard-expanded.