search

eqcorrscan / EQcorrscan

Earthquake detection and analysis in Python.
https://eqcorrscan.readthedocs.io/en/latest/
Other
166 stars 86 forks source link

if pick.phase_hint.upper()[0] == 'S'] AttributeError: 'NoneType' object has no attribute 'upper' #479

Closed fontiela closed 2 years ago

fontiela commented 3 years ago

When I'm running code to create tribe from QuakeMl at some point (after create several templates) I got the following error:

Traceback (most recent call last):
  File "/Users/joaofontiela/PycharmProjects/eq_location_eqcorrscan/tribo_qnet.py", line 17, in <module>
    tribe = Tribe().construct(method='from_meta_file', meta_file=quakeml, st=st, lowcut=2.0,
  File "/Users/joaofontiela/opt/anaconda3/envs/eqcorrscan/lib/python3.8/site-packages/eqcorrscan/core/match_filter/tribe.py", line 1016, in construct
    templates, catalog, process_lengths = template_gen.template_gen(
  File "/Users/joaofontiela/opt/anaconda3/envs/eqcorrscan/lib/python3.8/site-packages/eqcorrscan/core/template_gen.py", line 399, in template_gen
    template = _template_gen(
  File "/Users/joaofontiela/opt/anaconda3/envs/eqcorrscan/lib/python3.8/site-packages/eqcorrscan/core/template_gen.py", line 738, in _template_gen
    s_pick = [pick for pick in station_picks
  File "/Users/joaofontiela/opt/anaconda3/envs/eqcorrscan/lib/python3.8/site-packages/eqcorrscan/core/template_gen.py", line 739, in <listcomp>
    if pick.phase_hint.upper()[0] == 'S']
AttributeError: 'NoneType' object has no attribute 'upper'
flixha commented 3 years ago

It looks like the quakeml-event file that you are reading includes picks where no phase-hint is defined, so that the pick doesn't have that property once it's loaded. This can happen with more or less any property that one usually expects to be present in a catalog, that's probably why it's not taken into account in 100 % of cases. You can try to read in the event-catalog, exclude those picks that don't have a phase-hint (they're probably not useful if you don't know what they are), and run the template-creation with the fixed catalog-object approximately like this:

from obspy.core.event import read_events
cat = read_events(quakeml)
for event in cat:
    event.picks = [pick for pick in event.picks if pick.phase_hint is not None]
template_gen.template_gen(method='from_meta_file', ...., meta_file=cat, ...)
fontiela commented 3 years ago

@flixha thanks for your tip. It will make my life easy.

fontiela commented 3 years ago

@flixha and @calum-chamberlain Only yesterday I had time to re-take this subject. I inspected which earthquake lack phase hint with @flixha suggestion; and was in the last event of my list. I verify carefully this event in SEISAN and everything is fine. Then, I looked into the collect file, and it was fine also. To test, I removed this event from collect.out (create new qml with nor2qml) and run again EQCORRSCAN for my surprise was the last one. I removed this event repeating the same process and I got the same error. On my last test, I removed from my list (collect file) the events until the last event that EQCORRSCAN create template. I got the same error. So, I can resume problem is with the last event in quakexml. I'm not sure if the problem is in EQCORRSCAN or in the nor2qml. I'm looking for your support.

flixha commented 2 years ago

@fontiela Are you able to share some sample files, best the collect.out and the stationxml-file, and maybe add a minimal working example (I guess just 2-3 lines) of the code that produces the error?

fontiela commented 2 years ago

When I run the following code:

cat = read_events('/Users/joaofontiela/Documents/Campanhas/Terceira/QNET/Test/collect_test.xml')

for event in cat:
    event.picks = [pick for pick in event.picks if pick.phase_hint is not None]
print(event.picks)
templates = template_gen.template_gen(method='from_meta_file', meta_file=cat, lowcut=2.0,
                                      highcut=9.0, samp_rate=20.0, filt_order=3, length=1, prepick=0.1,swin='all',
                                      all_horiz=False, plot=True, plotdir=plots_dire)

I got the following message:

2021-11-29 19:26:26,866 eqcorrscan.core.template_gen    INFO    Pre-processing data
2021-11-29 19:26:26,866 eqcorrscan.core.template_gen    INFO    No data
[Pick(resource_id=ResourceIdentifier
 (id="smi:authorityid/Pick/2019-QS01-hdEO4nlfxHca"), time=UTCDateTime(2019, 8, 5, 17, 26, 28, 700000), waveform_id=WaveformStreamID(network_code='', station_code='QS01', channel_code='E?Z'), onset='impulsive', phase_hint='P', polarity='positive'), Pick(resource_id=ResourceIdentifier
 (id="smi:authorityid/Pick/2019-QS08-advlNpEkTqq5"), time=UTCDateTime(2019, 8, 5, 17, 26, 29, 770000), waveform_id=WaveformStreamID(network_code='', station_code='QS08', channel_code='E?N'), onset='impulsive', phase_hint='S', polarity='positive'), Pick(resource_id=ResourceIdentifier
 (id="smi:authorityid/Pick/2019-QG06-V2yAU99Vokwj"), time=UTCDateTime(2019, 8, 5, 17, 26, 28, 870000), waveform_id=WaveformStreamID(network_code='', station_code='QG06', channel_code='E?Z'), onset='impulsive', phase_hint='P', polarity='positive'), Pick(resource_id=ResourceIdentifier
 (id="smi:authorityid/Pick/2019-QG06-wXRBzmnKKtJj"), time=UTCDateTime(2019, 8, 5, 17, 26, 30, 190000), waveform_id=WaveformStreamID(network_code='', station_code='QG06', channel_code='E?N'), onset='emergent', phase_hint='S', polarity='negative'), Pick(resource_id=ResourceIdentifier
 (id="smi:authorityid/Pick/2019-QG06-61uLhB0j3XsV"), time=UTCDateTime(2019, 8, 5, 17, 26, 31, 40000), waveform_id=WaveformStreamID(network_code='', station_code='QG06', channel_code='E?Z'), phase_hint='IAML'), Pick(resource_id=ResourceIdentifier
 (id="smi:authorityid/Pick/2019-QB04-Dj0jNTc78r5U"), time=UTCDateTime(2019, 8, 5, 17, 26, 28, 980000), waveform_id=WaveformStreamID(network_code='', station_code='QB04', channel_code='H?Z'), onset='impulsive', phase_hint='P', polarity='positive'), Pick(resource_id=ResourceIdentifier
 (id="smi:authorityid/Pick/2019-QG03-ZIaFbvKxNQdu"), time=UTCDateTime(2019, 8, 5, 17, 26, 29, 940000), waveform_id=WaveformStreamID(network_code='', station_code='QG03', channel_code='E?Z'), onset='impulsive', phase_hint='P', polarity='positive'), Pick(resource_id=ResourceIdentifier
 (id="smi:authorityid/Pick/2019-QG03-y279A3dI8vtG"), time=UTCDateTime(2019, 8, 5, 17, 26, 31, 490000), waveform_id=WaveformStreamID(network_code='', station_code='QG03', channel_code='E?N'), onset='impulsive', phase_hint='S', polarity='positive'), Pick(resource_id=ResourceIdentifier
 (id="smi:authorityid/Pick/2019-QG03-hp24IC5hiPga"), time=UTCDateTime(2019, 8, 5, 17, 26, 31, 830000), waveform_id=WaveformStreamID(network_code='', station_code='QG03', channel_code='E?Z'), phase_hint='IAML'), Pick(resource_id=ResourceIdentifier
 (id="smi:authorityid/Pick/2019-QG04-la8mbdOnTjY6"), time=UTCDateTime(2019, 8, 5, 17, 26, 30, 310000), waveform_id=WaveformStreamID(network_code='', station_code='QG04', channel_code='E?N'), onset='impulsive', phase_hint='S', polarity='negative'), Pick(resource_id=ResourceIdentifier
 (id="smi:authorityid/Pick/2019-QB03-mWBiUBQlByzR"), time=UTCDateTime(2019, 8, 5, 17, 26, 30, 640000), waveform_id=WaveformStreamID(network_code='', station_code='QB03', channel_code='H?Z'), onset='impulsive', phase_hint='P', polarity='positive'), Pick(resource_id=ResourceIdentifier
 (id="smi:authorityid/Pick/2019-QB03-XDAzaVMFfLgU"), time=UTCDateTime(2019, 8, 5, 17, 26, 32, 340000), waveform_id=WaveformStreamID(network_code='', station_code='QB03', channel_code='H?N'), onset='impulsive', phase_hint='S', polarity='positive'), Pick(resource_id=ResourceIdentifier
 (id="smi:authorityid/Pick/2019-QG04-RSU4KhnLhs0g"), time=UTCDateTime(2019, 8, 5, 17, 26, 30, 620000), waveform_id=WaveformStreamID(network_code='', station_code='QG04', channel_code='E?Z'), phase_hint='IAML'), Pick(resource_id=ResourceIdentifier
 (id="smi:authorityid/Pick/2019-QG04-ipV2IxnJzOo9"), time=UTCDateTime(2019, 8, 5, 17, 26, 29, 820000), waveform_id=WaveformStreamID(network_code='', station_code='QG04', channel_code='E?Z'), onset='impulsive', phase_hint='P', polarity='negative'), Pick(resource_id=ResourceIdentifier
 (id="smi:authorityid/Pick/2019-QS08-UIxwSXMwH1Qw"), time=UTCDateTime(2019, 8, 5, 17, 26, 28, 620000), waveform_id=WaveformStreamID(network_code='', station_code='QS08', channel_code='E?Z'), onset='impulsive', phase_hint='P', polarity='positive'), Pick(resource_id=ResourceIdentifier
 (id="smi:authorityid/Pick/2019-QS08-8BhNTSF97MAR"), time=UTCDateTime(2019, 8, 5, 17, 26, 30, 530000), waveform_id=WaveformStreamID(network_code='', station_code='QS08', channel_code='E?Z'), phase_hint='IAML')]

Find attached the collect.out file (from SEISAN 11.0) and QuakeXML (nor2qml)

With the same dataset, when I try create a tribe with this code:

for filename in glob.glob(TEST_PATH + '/WAV_TEST1/' + "2019*"):
    st = read(filename)
    tribe = Tribe().construct(method='from_meta_file', meta_file=quakeml, st=st, lowcut=2.0,
                         highcut=9.0, samp_rate=20.0, filt_order=4, length=2, prepick=0.1,
                         swin='S_all', all_horiz=False, plot=True, plotdir=plots_dire)

I got a new error relate with phase-hint that wasn't identified by the first code snippet:

2021-11-29 19:37:37,326 eqcorrscan.core.template_gen    DEBUG   Pick
     resource_id: ResourceIdentifier(id="smi:authorityid/Pick/2019-QG03-ImU1AZhip6Wx")
            time: UTCDateTime(2019, 8, 5, 8, 23, 27, 630000)
     waveform_id: WaveformStreamID(network_code='', station_code='QG03', channel_code='E?E')
           onset: 'impulsive'
      phase_hint: 'S'
        polarity: 'positive'
2021-11-29 19:37:37,326 eqcorrscan.core.template_gen    DEBUG   Pick
     resource_id: ResourceIdentifier(id="smi:authorityid/Pick/2019-QG03-02ByFYcK8AWX")
            time: UTCDateTime(2019, 8, 5, 8, 23, 28, 490000)
     waveform_id: WaveformStreamID(network_code='', station_code='QG03', channel_code='E?Z')
      phase_hint: 'IAML'
2021-11-29 19:37:37,326 eqcorrscan.core.template_gen    DEBUG   Pick
     resource_id: ResourceIdentifier(id="smi:authorityid/Pick/2019-QG04-P7Rkzeo4HvFV")
            time: UTCDateTime(2019, 8, 5, 8, 23, 26, 500000)
     waveform_id: WaveformStreamID(network_code='', station_code='QG04', channel_code='E?Z')
           onset: 'impulsive'
      phase_hint: 'P'
        polarity: 'negative'
Traceback (most recent call last):
  File "/Users/joaofontiela/PycharmProjects/eq_location_eqcorrscan/tribo_qnet.py", line 19, in <module>
    tribe = Tribe().construct(method='from_meta_file', meta_file=quakeml, st=st, lowcut=2.0,
  File "/Users/joaofontiela/opt/anaconda3/envs/eqcorrscan/lib/python3.8/site-packages/eqcorrscan/core/match_filter/tribe.py", line 1016, in construct
    templates, catalog, process_lengths = template_gen.template_gen(
  File "/Users/joaofontiela/opt/anaconda3/envs/eqcorrscan/lib/python3.8/site-packages/eqcorrscan/core/template_gen.py", line 399, in template_gen
    template = _template_gen(
  File "/Users/joaofontiela/opt/anaconda3/envs/eqcorrscan/lib/python3.8/site-packages/eqcorrscan/core/template_gen.py", line 738, in _template_gen
    s_pick = [pick for pick in station_picks
  File "/Users/joaofontiela/opt/anaconda3/envs/eqcorrscan/lib/python3.8/site-packages/eqcorrscan/core/template_gen.py", line 739, in <listcomp>
    if pick.phase_hint.upper()[0] == 'S']
AttributeError: 'NoneType' object has no attribute 'upper'

collect.out.zip collect_test.xml.zip

calum-chamberlain commented 2 years ago

Just a quick one:

  1. Sorry, I have been in the field hence my unresponsiveness;
  2. If this is the code you are running then you are missing providing any data for the code to make the template from - see the note in the docs here about the from_meta_file method which requires an st argument.

The No data message from the first example stems from this line which is because you haven't provided any data.

In the second example, at least one of your events has a pick without a phase hint. Event ID 20190803224220 has a pick for station QG03 that is emergent, but has no phase hint associated with it. Picks like this are likely the source of the second error, which is something that should be fixed in EQcorrscan (I should change this error to a warning). The pick I found is this line in the collect.out:

 QG03 EZ E      D 2243 24.27                                           96.8 308
calum-chamberlain commented 2 years ago

Missing phase-hints should be handled in #510. Closing.