Aurora not passing since merge

[kkappler]

Bumping into a few issues since main was merged into fcs last week.

Currently can't write tf as xml due to the following error:

Traceback (most recent call last): File "test_define_frequency_bands.py", line 67, in test() File "test_define_frequency_bands.py", line 63, in test test_can_declare_frequencies_directly_in_config() File "test_define_frequency_bands.py", line 56, in test_can_declare_frequencies_directly_in_config tf_cls1.write(fn="cfg1.xml", file_type="emtfxml") File "/home/kkappler/software/irismt/mt_metadata/mt_metadata/transfer_functions/core.py", line 1248, in write obj = self._read_write_dict[file_type]["write"]() File "/home/kkappler/software/irismt/mt_metadata/mt_metadata/transfer_functions/core.py", line 1423, in to_emtfxml emtf.station_metadata = self.station_metadata File "/home/kkappler/software/irismt/mt_metadata/mt_metadata/base/metadata.py", line 401, in setattr super().setattr(name, value) File "/home/kkappler/software/irismt/mt_metadata/mt_metadata/transfer_functions/io/emtfxml/emtfxml.py", line 1088, in station_metadata if rch.sensor.name in ["NIMS", "LEMI"] and rch.sensor.type in [ AttributeError: 'NoneType' object has no attribute 'sensor'

I traced this back a ways ... it was not happening because of something wrong in the sensor config ... rather it is happening in the for loop around line 1055 in emtfxml.py where iteration is done over sm.runs

sm is mt_metadata.transfer_functions.tf.station.Station,

The issue is probably related to how I am building the tf object in aurora, which is done in process_mth5.py in the function export_tf()

When I encountered the error I was initializing the station_metadata._runs as a list,

tf_cls.station_metadata._runs = [ ]
tf_cls.station_metadata.from_dict(station_metadata_dict)

In that case tf_cls.station_metadata.runs has two runs, but it should only have one.

OrderedDict([(None, { "run": { "channels_recorded_auxiliary": [], "channels_recorded_electric": [ "ex", "ey" ], "channels_recorded_magnetic": [ "hx", "hy", "hz" ], "data_logger.firmware.author": null, "data_logger.firmware.name": null, "data_logger.firmware.version": null, "data_logger.id": null, "data_logger.manufacturer": null, "data_logger.timing_system.drift": 0.0, "data_logger.timing_system.type": "GPS", "data_logger.timing_system.uncertainty": 0.0, "data_logger.type": null, "data_type": "BBMT", "hdf5_reference": null, "id": null, "mth5_type": null, "sample_rate": 0.0, "time_period.end": "1980-01-01T00:00:00+00:00", "time_period.start": "1980-01-01T00:00:00+00:00" } }), ('001', { "run": { "channels_recorded_auxiliary": [], "channels_recorded_electric": [], "channels_recorded_magnetic": [], "data_logger.firmware.author": null, "data_logger.firmware.name": null, "data_logger.firmware.version": null, "data_logger.id": null, "data_logger.manufacturer": null, "data_logger.timing_system.drift": 0.0, "data_logger.timing_system.type": "GPS", "data_logger.timing_system.uncertainty": 0.0, "data_logger.type": null, "data_type": "BBMT", "hdf5_reference": null, "id": "001", "mth5_type": null, "sample_rate": 0.0, "time_period.end": "1980-01-01T00:00:00+00:00", "time_period.start": "1980-01-01T00:00:00+00:00" } })])

The first entry has channels_recorded assigned, and does not crash, but the second entry does crash (presumably because channels_recorded is empty).

I changed to initializing with an mt_metadata ListDict

tf_cls.station_metadata._runs = ListDict()
tf_cls.station_metadata.from_dict(station_metadata_dict)

and in that case I correctly get only 1 run: tf_cls.station_metadata.run:

OrderedDict([('001', { "run": { "channels_recorded_auxiliary": [], "channels_recorded_electric": [], "channels_recorded_magnetic": [], "data_logger.firmware.author": null, "data_logger.firmware.name": null, "data_logger.firmware.version": null, "data_logger.id": null, "data_logger.manufacturer": null, "data_logger.timing_system.drift": 0.0, "data_logger.timing_system.type": "GPS", "data_logger.timing_system.uncertainty": 0.0, "data_logger.type": null, "data_type": "BBMT", "hdf5_reference": null, "id": "001", "mth5_type": null, "sample_rate": 0.0, "time_period.end": "1980-01-01T00:00:00+00:00", "time_period.start": "1980-01-01T00:00:00+00:00" } })])

Some manual assignment of tf_cls.station_metadata.run.channels_recorded_magnetic = ["hx", "hy", "hz"] seems to get my by this error, so the question is how to get those values assigned inside: tf_cls.station_metadata.from_dict(station_metadata_dict)

given the the input station_metadata_dict has the form:

{'station': OrderedDict([('acquired_by.name', None), ('channels_recorded', ['ex', 'ey', 'hx', 'hy', 'hz']), ('data_type', 'BBMT'), ('geographic_name', None), ('hdf5_reference', ''), ('id', 'test1'), ('location.declination.model', 'WMM'), ('location.declination.value', 0.0), ('location.elevation', 0.0), ('location.latitude', 0.0), ('location.longitude', 0.0), ('mth5_type', 'Station'), ('orientation.method', None), ('orientation.reference_frame', 'geographic'), ('provenance.creation_time', '1980-01-01T00:00:00+00:00'), ('provenance.software.author', None), ('provenance.software.name', None), ('provenance.software.version', None), ('provenance.submitter.email', None), ('provenance.submitter.organization', None), ('release_license', 'CC0-1.0'), ('run_list', ['001']), ('time_period.end', '1980-01-01T00:00:00+00:00'), ('time_period.start', '1980-01-01T00:00:00+00:00')])}

i.e. it has a list of recorded stations ...

This maybe related to the issue I am encountering with zmm files in aurora also.

[kkappler]

@kujaku11 If you take a look at this, a couple things:

1. The aurora branch fix_issue_143 has already got some updates, such as replacing write_tf_file() with write() for the xml writer.
2. A relatively quick aurora test that shows this problem is aurora/tests/synthetic/test_define_frequency_bands.py