kujaku11 / mth5

Exchangeable and archivable format for magnetotelluric time series to better serve the community through FAIR principles.
https://mth5.readthedocs.io/en/latest/index.html
MIT License
15 stars 3 forks source link

Cannot Convert FDSN Channel to MTH5 Channel Back to FDSN Channel #54

Open timronan opened 2 years ago

timronan commented 2 years ago

FDSN channel: networks = ['BK'] stations = ['PKD'] channels = {'BQ2':''} starttime = '2004-09-28T00:00:00.000000' endtime = '2004-09-28T01:59:59.974999'

Is being collected through the IRIS web client and it's metadata is converted to an MTH5 framework using:

translator = XMLInventoryMTExperiment()
experiment = translator.xml_to_mt(inv)

When trying to convert the electric channel back to an FDSN channe code using 'fdsn_tools.make_channel_code(mth5_chan)`

The error below is thrown.

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-3-a6645837fb05> in <module>
----> 1 MakeMTH5.make_mth5_from_fdsnclient(networks, stations, channels, starttime, endtime, path=None, client="IRIS")

~/Desktop/Projects/mth5/mth5/clients/make_mth5.py in make_mth5_from_fdsnclient(networks, stations, channels, starttime, endtime, path, client)
     97                     mth5_chan = m.stations_group.get_station(msta_id)\
     98                                  .get_run(mrun_id).get_channel(mchan_id)
---> 99                     print(fdsn_tools.make_channel_code(mth5_chan))
    100                     # Map the channel but mth5 and ph5
    101 

~/Desktop/Projects/mth5/mth5/utils/fdsn_tools.py in make_channel_code(channel_obj)
    166 
    167     period_code = get_period_code(channel_obj.sample_rate)
--> 168     sensor_code = get_measurement_code(channel_obj.type)
    169     if "z" in channel_obj.component.lower():
    170         orientation_code = get_orientation_code(

AttributeError: 'ElectricDataset' object has no attribute 'type'

Upon inspection Channel Electrics does not seem to have a type:

Channel Electric:
-------------------
    component:        ey
    data type:        electric
    data format:      int32
    data shape:       (1,)
    start:            2003-09-12T18:54:00+00:00
    end:              2005-03-15T16:45:00+00:00
    sample rate:      40.0

Is this coming from how the channel is built or because this is missing from the original stationxml.

kujaku11 commented 2 years ago

@timronan I think that instead of passing the mth5_channel, which is a h5py.Dataset you need to pass the metadata object associated with it. mth5_chan.metadata, this should pass on the appropriate metadata. Maybe this is a bit obtuse and needs some rethinking, but that's how it works for now, I think.

mth5_chan = m.stations_group.get_station(msta_id).get_run(mrun_id).get_channel(mchan_id)
print(fdsn_tools.make_channel_code(mth5_chan.metadata))
timronan commented 2 years ago

I jut needed the .metadata part. Thanks for the clarification.

timronan commented 2 years ago

The channel code is getting converted when going from FDSN to MTH5 to FDSN: BQ2 to BQE and BQ3 to BQZ

Input: BQ2
Channel Electric:
-------------------
    component:        ey
    data type:        electric
    data format:      int32
    data shape:       (1,)
    start:            2003-09-12T18:54:00+00:00
    end:              2005-03-15T16:45:00+00:00
    sample rate:      40.0
Ouput: BQE
Input: BQ3
Channel Electric:
-------------------
    component:        ez
    data type:        electric
    data format:      int32
    data shape:       (1,)
    start:            2003-09-12T18:54:00+00:00
    end:              2005-03-15T16:45:00+00:00
    sample rate:      40.0
Ouput: BQZ

Do we want this transformation? I can deal with it using a dictionary but want to make sure we want the channel codes automatically transformed.

kujaku11 commented 2 years ago

@timronan I coded it up to follow the FDSN guidelines on N, E, Z, 1, 2, 3, which are based on the azimuth. There is a 15 degree? buffer to N, E, and Z, otherwise they are labelled with 1, 2, 3. I'm not sure how to make this robust unless we stick with 1, 2, 3...., which has more freedom. Thoughts?

timronan commented 2 years ago

The SEED manual states :

When orthogonal directions are used, there are traditional orientations of North (N), East (E), and Vertical (Z), as well as other orientations that can readily be converted to traditional ones. These options are detailed with each instrument type. Use N or E for the orientation when it is within 5 degrees of north or east. Use 1 or 2 when orientations are more than 5 degrees from north or east.

Magnetometer: Measures the magnetic field where the instrument is sitting. They measure the part of the field vector that is aligned with the measurement coil. Many magnetometers are three axis. The instrument will typically be oriented to local magnetic north. The dip and azimuth should describe this in terms of the geographic north. Example: Local magnetic north is 13 degrees east of north in Albuquerque. So if the magnetometer is pointed to magnetic north, the azimuth would be + 103 for the E channel. Some magnetometers do not record any vector quantity associated with the signal, but record the total intensity. So, these would not have any dip/ azimuth. Instrument Code F Orientation Code Z N E Magnetic Signal Units: T — Teslas Channel Flags: G

Electric Potential: Measures the Electric Potential between two points. This is normally done using a high impedance voltmeter connected to two electrodes driven into the ground. In the case of magnetotelleuric work, this is one parameter that must be measured. Instrument Code Q Orientation Code Unknown Signal Units: V — Volts Channel Flags: G

Most of the time, it seems better to be transparent when transforming data between containers, and not changing the orientation codes, but there are cases when it makes sense to normalize a specific data repository to a set of standards that were previously not defined/ignored. The question is, how strict should MTH5 be with channel code? If we want to create a uniform data repository we should potentially transform the data, but if we want to preserve a historical standard we should not alter the channel codes if possible. Do MT systems follow a similar orthogonal as seismometers?

akelbert commented 2 years ago

@kujaku11 @timronan @kkappler as you all know, when the E-field channels are modified and new epochs appear, the codes are LQ1 & LQ2 in place of LQN and LQE that we're used to. This is probably fine except that my miniseed creation code doesn't support this. According to Jared:


For orthogonal components that are in nontraditional orientations, if the orientation of the horizontal components is known to deviate more than 5 degrees from true North and East, the respective channels should be named 1, 2 instead of N, E (N->1, E->2).  

Channel codes — FDSN Source Identifiers

Which means that when we put dipoles into geographic coordinates the angle is usually > 10 from North or east and therefore the direction identifier becomes a number.  Now I don't know that this is a hard definition or more guidelines.  I gave it a 15 degree variance but I guess that isn't enough. We could change it to 30 or 45 I guess that should keep X and Y in nearly all cases.  

The code that controls this behavior is in mt_metadata/mt_metadata/timeseries/stationxml/fdsn_tools.py [lines 77-84] and lines[148-179]. 

I think it would be important to make sure that the naming of electric field channels is consistent in ALL cases (rather than most). Magnetic as well of course but that seems less of a problem at present. The software that I use to create miniseeds was written by Chad Trabant at IRIS in C++, and it's hardcoded to produce LQN and LQE. I can change that of course, but creating logic to vary these names depending on orientation would be extremely complex. In fact the software that makes MT miniseeds currently has no access to metadata, and shouldn't, in principle... that's the whole point of miniseeds.

Are we allowed to hard-code x=N and y=E? If we are, that seems like the cleanest solution for this. Otherwise, perhaps we can hard-code x=1 and y=2. Either way we need a consistent solution for archiving. Thoughts?