swiss-seismological-service / scdetect

A computationally efficient earthquake detection module for SeisComP
https://scdetect.readthedocs.io
GNU Affero General Public License v3.0
15 stars 6 forks source link

minimumArrivals #101

Closed mmesim closed 2 years ago

mmesim commented 2 years ago

@damb Well, I shared with you the files. Here I'll report the problem. I run the shared example and set minimumArrivals: 10. However, when I run scevent with the results.scml I received the following message multiple times

rejecting automatic origin Origin/20220318161026.734837.31225 (phaseCount: 3 < 10)

or

rejecting automatic origin Origin/20220318161026.734837.31225 (phaseCount: 6 < 10)

That probably means that there are not 10 picks available, which seems weird.

I still do not if it scdetect-cc or scevent problem.

I didn't include the results.scml to check whether you can fully reproduce the error.

damb commented 2 years ago

@mmesim, thanks for reporting.

That probably means that there are not 10 picks available, which seems weird.

Without having checked the reproducer's configuration yet, I guess that this might be an issue when trying to avoid to associate a single phase to be associated multiple times with a sensor location. This is required in certain configuration scenarios, e.g. given the following detector configuration:

  {
    "detectorId": "detector-01",
    "createArrivals": true,
    "originId": "originId",
    "templatePhase": "Sg",
    "templateWaveformStart": -0.25,
    "templateWaveformEnd": 2,
    "streams": [
      {
        "templateId": "template-01",
        "waveformId": "CH.GRIMS..HHN"
      },
      {
        "templateId": "template-02",
        "waveformId": "CH.GRIMS..HHE"
      }
    ]
  }

I.e. in the example above, in order to avoid to associate an Arrival with phase Sg twice (regarding the sensor location CH.GRIMS..) scdetect-cc drops one of the arrivals including the respective pick. This is not ideal, however, AFAIU, in the SeisComP datamodel the relationship Pick <-> Origin can be established only by means of an Arrival.

Do you have a better idea how to circumvent this issue?

mmesim commented 2 years ago

What I do not understand is why I didn't face this problem with the other testing datasets.

If what you mention above is the case, then it should have happened before. Still thinking about it...

damb commented 2 years ago

What I do not understand is why I didn't face this problem with the other testing datasets.

I'll have a look into the reproducer and come back.

damb commented 2 years ago

If what you mention above is the case, then it should have happened before. Still thinking about it...

mmesim commented 2 years ago

You mean, while using the "minimumArrivals" configration parameter, as well?

Yes.

Did you check scevent logs previously, too?

Yes

damb commented 2 years ago

Ok. So when trying to initialize detector-01 with

$ mkdir data.sds && \
scart -I data.mseed data.sds && \
scdetect-cc --templates-prepare --debug --offline --playback --ep=detections.scml --templates-json=$(realpath templates.json) --inventory-db="file://$(realpath inventory.scml)" --record-url="sdsarchive://$(realpath data.sds)" --event-db="file://$(realpath catalog.scml)" --config-db="file://$(realpath config.scml)"   --amplitudes-force=0

I obtain the following logs:

Logs ``` 14:36:45 [debug] Adding plugin path: . 14:36:45 [debug] Adding plugin path: /home/damb/.seiscomp/plugins 14:36:45 [debug] Adding plugin path: /home/damb/seiscomp/lib/plugins 14:36:45 [debug] Adding plugin path: /home/damb/seiscomp/lib 14:36:45 [debug] Adding plugin path: /home/damb/seiscomp/share/plugins 14:36:45 [debug] Adding plugin path: /home/damb/seiscomp/share/plugins/scdetect-cc 14:36:45 [debug] Trying to open plugin at /home/damb/seiscomp/share/plugins/dbpostgresql.so 14:36:45 [info] Plugin dbpostgresql registered 14:36:45 [info] Plugins: -------- [1] description: PostgreSQL database driver author: GFZ Potsdam version: 0.11.0 API: 15.0.0 14:36:45 [info] Loading complete inventory from file:///home/damb/tmp/scdetect/iss-101/stations.scml 14:36:45 [info] Finished loading complete inventory 14:36:45 [debug] trying to open stream file:///home/damb/tmp/scdetect/iss-101/data.mseed 14:36:45 [info] Playback mode enabled 14:36:45 [info] Loading events from file:///home/damb/tmp/scdetect/iss-101/events.scml 14:36:45 [debug] Found database version v0.12 14:36:46 [debug] Found database version v0.12 14:36:46 [info] Finished loading events 14:36:46 [info] Loading template configuration from /home/damb/tmp/scdetect/iss-101/templates.json 14:36:46 [debug] Creating detector processor (id=detector-01) ... 14:36:46 [debug] [8D.ELM1..EHZ (8D.ELM1..EHZ)] using arrival pick: origin=smi:ch.ethz.sed/sc3a/origin/NLL.20201111122232.712663.968378, time=2020-10-25T19:38:39.014947Z, phase=Pg, stream=8D.ELM1..EHZ 14:36:46 [debug] [8D.ELM1..EHZ (8D.ELM1..EHZ)] loaded stream from inventory for epoch: start=2020-10-25T19:38:38.764948Z, end=2020-10-25T19:38:40.014947Z 14:36:46 [debug] [8D.ELM1..EHZ (8D.ELM1..EHZ)] creating template waveform processor with id: detector-01::template-10 14:36:46 [debug] [detector-01::template-10] [8D.ELM1..EHZ (8D.ELM1..EHZ)] filters configured: filter="BW_BP(2,1.5,15)" 14:36:46 [warning] [CH.STLM2..HGN (CH.STLM2..HGN)] sensor location not found in inventory for time: 2020-10-25T19:38:48.784752Z. Skipping template waveform processor initialization. 14:36:46 [warning] [CH.STLM2..HGZ (CH.STLM2..HGZ)] sensor location not found in inventory for time: 2020-10-25T19:38:37.967674Z. Skipping template waveform processor initialization. 14:36:46 [debug] [CH.PANIX..HHN (CH.PANIX..HHN)] using arrival pick: origin=smi:ch.ethz.sed/sc3a/origin/NLL.20201111122232.712663.968378, time=2020-10-25T19:38:40.111906Z, phase=Sg, stream=CH.PANIX..HHE 14:36:46 [debug] [CH.PANIX..HHN (CH.PANIX..HHN)] loaded stream from inventory for epoch: start=2020-10-25T19:38:39.861907Z, end=2020-10-25T19:38:42.111906Z 14:36:46 [debug] [CH.PANIX..HHN (CH.PANIX..HHN)] creating template waveform processor with id: detector-01::template-06 14:36:46 [debug] [detector-01::template-06] [CH.PANIX..HHN (CH.PANIX..HHN)] filters configured: filter="BW_BP(2,1.5,15)" 14:36:46 [debug] [CH.PANIX..HHE (CH.PANIX..HHE)] using arrival pick: origin=smi:ch.ethz.sed/sc3a/origin/NLL.20201111122232.712663.968378, time=2020-10-25T19:38:40.111906Z, phase=Sg, stream=CH.PANIX..HHE 14:36:46 [debug] [CH.PANIX..HHE (CH.PANIX..HHE)] loaded stream from inventory for epoch: start=2020-10-25T19:38:39.861907Z, end=2020-10-25T19:38:42.111906Z 14:36:46 [debug] [CH.PANIX..HHE (CH.PANIX..HHE)] creating template waveform processor with id: detector-01::template-05 14:36:46 [debug] [detector-01::template-05] [CH.PANIX..HHE (CH.PANIX..HHE)] filters configured: filter="BW_BP(2,1.5,15)" 14:36:46 [debug] [CH.PANIX..HHZ (CH.PANIX..HHZ)] using arrival pick: origin=smi:ch.ethz.sed/sc3a/origin/NLL.20201111122232.712663.968378, time=2020-10-25T19:38:38.84245Z, phase=Pg, stream=CH.PANIX..HHZ 14:36:46 [debug] [CH.PANIX..HHZ (CH.PANIX..HHZ)] loaded stream from inventory for epoch: start=2020-10-25T19:38:38.592451Z, end=2020-10-25T19:38:39.84245Z 14:36:46 [debug] [CH.PANIX..HHZ (CH.PANIX..HHZ)] creating template waveform processor with id: detector-01::template-04 14:36:46 [debug] [detector-01::template-04] [CH.PANIX..HHZ (CH.PANIX..HHZ)] filters configured: filter="BW_BP(2,1.5,15)" 14:36:46 [debug] [8D.ELM1..EHN (8D.ELM1..EHN)] using arrival pick: origin=smi:ch.ethz.sed/sc3a/origin/NLL.20201111122232.712663.968378, time=2020-10-25T19:38:40.331748Z, phase=Sg, stream=8D.ELM1..EHE 14:36:46 [debug] [8D.ELM1..EHN (8D.ELM1..EHN)] loaded stream from inventory for epoch: start=2020-10-25T19:38:40.081749Z, end=2020-10-25T19:38:42.331748Z 14:36:46 [debug] [8D.ELM1..EHN (8D.ELM1..EHN)] creating template waveform processor with id: detector-01::template-12 14:36:46 [debug] [detector-01::template-12] [8D.ELM1..EHN (8D.ELM1..EHN)] filters configured: filter="BW_BP(2,1.5,15)" 14:36:46 [debug] [8D.ELM1..EHE (8D.ELM1..EHE)] using arrival pick: origin=smi:ch.ethz.sed/sc3a/origin/NLL.20201111122232.712663.968378, time=2020-10-25T19:38:40.331748Z, phase=Sg, stream=8D.ELM1..EHE 14:36:46 [debug] [8D.ELM1..EHE (8D.ELM1..EHE)] loaded stream from inventory for epoch: start=2020-10-25T19:38:40.081749Z, end=2020-10-25T19:38:42.331748Z 14:36:46 [debug] [8D.ELM1..EHE (8D.ELM1..EHE)] creating template waveform processor with id: detector-01::template-11 14:36:46 [debug] [detector-01::template-11] [8D.ELM1..EHE (8D.ELM1..EHE)] filters configured: filter="BW_BP(2,1.5,15)" 14:36:46 [warning] [8D.EML0..EHN (8D.EML0..EHN)] sensor location not found in inventory for time: 2020-10-25T19:38:48.784752Z. Skipping template waveform processor initialization. 14:36:46 [warning] [8D.EML0..EHE (8D.EML0..EHE)] sensor location not found in inventory for time: 2020-10-25T19:38:48.784752Z. Skipping template waveform processor initialization. 14:36:46 [warning] [CH.STLM2..HGE (CH.STLM2..HGE)] sensor location not found in inventory for time: 2020-10-25T19:38:48.784752Z. Skipping template waveform processor initialization. 14:36:46 [warning] [8D.EML0..EHZ (8D.EML0..EHZ)] sensor location not found in inventory for time: 2020-10-25T19:38:37.967674Z. Skipping template waveform processor initialization. 14:36:46 [debug] Application initialized 14:36:46 [debug] Subscribing to streams required for processing 14:36:46 [debug] Subscribing to stream: 8D.ELM1..EHE 14:36:46 [debug] Subscribing to stream: 8D.ELM1..EHN 14:36:46 [debug] Subscribing to stream: 8D.ELM1..EHZ 14:36:46 [debug] Subscribing to stream: CH.PANIX..HHE 14:36:46 [debug] Subscribing to stream: CH.PANIX..HHN 14:36:46 [debug] Subscribing to stream: CH.PANIX..HHZ 14:36:46 [info] Starting record acquisition 14:36:46 [debug] [detector-01::template-11] Initialize stream: sampling_frequency=200.000000 14:36:46 [debug] [detector-01::template-12] Initialize stream: sampling_frequency=200.000000 14:36:46 [debug] [detector-01::template-10] Initialize stream: sampling_frequency=200.000000 14:36:46 [debug] [detector-01::template-05] Initialize stream: sampling_frequency=200.000000 14:36:46 [debug] [detector-01::template-04] Initialize stream: sampling_frequency=200.000000 14:36:46 [debug] [detector-01::template-06] Initialize stream: sampling_frequency=200.000000 14:36:48 [debug] [detector-01] Detector result (triggering) score=1.000000, associated_results=6 ^C14:36:50 [debug] [detector-01] Terminating ... 14:36:50 [debug] [detector-01] Start processing detection (time=2020-10-25T19:38:37.068104Z, associated_results=6) ... 14:36:50 [debug] Found 1 origins. 14:36:50 [debug] Leaving ::done 14:36:50 [info] Waiting for record thread 14:36:50 [debug] Unload plugin 'PostgreSQL database driver' ```

The warnings

14:36:46 [warning] [CH.STLM2..HGN (CH.STLM2..HGN)] sensor location not found in inventory for time: 2020-10-25T19:38:48.784752Z. Skipping template waveform processor initialization.
14:36:46 [warning] [CH.STLM2..HGZ (CH.STLM2..HGZ)] sensor location not found in inventory for time: 2020-10-25T19:38:37.967674Z. Skipping template waveform processor initialization.
14:36:46 [warning] [8D.EML0..EHN (8D.EML0..EHN)] sensor location not found in inventory for time: 2020-10-25T19:38:48.784752Z. Skipping template waveform processor initialization.
14:36:46 [warning] [8D.EML0..EHE (8D.EML0..EHE)] sensor location not found in inventory for time: 2020-10-25T19:38:48.784752Z. Skipping template waveform processor initialization.
14:36:46 [warning] [CH.STLM2..HGE (CH.STLM2..HGE)] sensor location not found in inventory for time: 2020-10-25T19:38:48.784752Z. Skipping template waveform processor initialization.
14:36:46 [warning] [8D.EML0..EHZ (8D.EML0..EHZ)] sensor location not found in inventory for time: 2020-10-25T19:38:37.967674Z. Skipping template waveform processor initialization.

indicate that the inventory metadata was not available for the corresponding streams and time window. The detector is initialized, anyway, however, just with 6 streams. Due to the behaviour described previously (i.e. in https://github.com/damb/scdetect/issues/101#issuecomment-1073745094), only 4 picks will be associated in case of a detection.

Does this help?

mmesim commented 2 years ago

I just realized that there was a typo. Instead of writing ELM0 I wrote EML0. :face_exhaling:

Also, I do not know why STLM2.HG? is not in the inventory. Perhaps I did not include acc? I don't remember.

To conclude, in the case that scdetect-cc will not be able to identify 10 phases, what will it do and how is this trackable/manageable in real time application?

damb commented 2 years ago

To conclude, in the case that scdetect-cc will not be able to identify 10 phases, what will it do and how is this trackable/manageable in real time application?

Warnings are already issued. Though, in case the "minimumArrivals" is configured and the value is greater than the number of streams available scdetect-cc may not initialize the detector at all. What do you suggest?

EDIT: Currently, scdetect-cc does initialize the detector, but it resets the "minimumArrivals" to the number of streams available.

mmesim commented 2 years ago

Need to think about it.

damb commented 2 years ago

Ok, this should be solved. Thanks for the discussion, @mmesim.

As a small enhancement, we're going to implement #102.