Should the self-detection time of a template be independent of other templates in the tribe?

I ran some tests to let an event template detect itself in continuous data. But depending on the station choice and what other templates were in the tribe that I used, I got different detection times (11 s apart). I tried to debug this and believe I have an idea of what's happening. I'm wondering whether this is the expected behaviour or not. If it's not expected, it would be nice if you have some feedback for how to solve it.

Issue description:

Let's detect an event with a template of itself in continuous data. There are two templates, A and B, with template A being created from the event that we want to detect itself.
- the earliest trace (i.e., pick) in the templates is for station DRUM, but for that station there is less than 80 % data during that day
- template B contains a pick + trace for station INVG, but there is not such pick or trace for template A
- both templates contain pick+trace for stations EKB10.BHZ and ESK.BHZ, but template B contains EKB10.BHZ twice because it's a 1-comp station with P and S-pick on the same channel
In case 1
- I use a tribe of two templates, A and B , with template A created from the event itself.
- for template A, eqcorrscan.utils.preprocessing._prep_data_for_correlation removes the traces for DRUM and adds two NaN-traces (for EKB10.BHZ and INVG.BHZ, with both traces starting at 18:14:42
- the stream contains traces EKB10.BHZ x2, ESK.BHZ, INVG.BHZ
- the event is detected at 18:14:42 (hh:mm:ss, same a P-pick on DRUM), and the detection-channel list contains [('EKB10', 'BHZ'), ('ESK', 'BHE'), ('ESK', 'BHN'), ('ESK', 'BHZ')]
In case 2
- I use a tribe of one template, A, created from the event itself.
- for template A eqcorrscan.utils.preprocessing._prep_data_for_correlation removes the trace for DRUM and INVG, so that the earliest remaining trace starts at 18:14:53
- the stream contains traces EKB10.BHZ, ESK.BHZ
- the event is detected at 18:14:53 (hh:mm:ss, i.e., P-pick on EKB10), and the detection-channel list mentions [('EKB10', 'BHZ'), ('ESK', 'BHE'), ('ESK', 'BHN'), ('ESK', 'BHZ')]

Analysis / Consequences:

So as I understand, the difference in detection time is because in case 2 with only one template for detection, there are no NaN-channels in the template. Hence the earliest trace in the template starts at 18:14:53.0 and that is then the detection time. If I try to use the detection from case 2 in lag_calc, I get the wrong picks. So I would say that also in case 2, we should get 18:14:42 as detection time, because the detection time should not change because there are other templates in the tribe or not.

Possible solution:

As a possible solution, one could do a check at the end of the template-preparation in eqcorrscan.utils.preprocessing._prep_data_for_correlation that does as follows:

if the earliest trace of the original template is to be removed because there is no continuous data for it
then don't remove it, but instead add a NaN-channel for that station-channel to the stream

Morning @flixha - I think your conclusion is probably right - that this is due to the earliest trace being thrown out completely in one case, and being converted to NaN in the other due to poor data quality in the continuous data. Having a different result depending on what templates are used is certainly not what we want, nor is getting the wrong pick times from lag_calc! Thanks for spotting this!

Your solution is certainly workable, but adds quite a lot of computational overhead and so probably isn't the most desirable way to fix this long-term, but if it helps you get your case up and running then go for it on your local code.

I think this should be fixed fairly urgently as it is something that I think would not be obvious and could be very frustrating for many people. @flixha would you be able to start a PR with a test-case that demonstrates this, e.g. a test that runs the single template and two template tribes and checks that they have the same detection times (this test should fail), then we can work on a solution in that PR.

Hi @calum-chamberlain, thanks for your quick reply! I will create and upload the test case soon (guess on Monday), and yes I can start the PR for that where we can work on the fix. I also thought that the NaN-trace is not optimal from the overhead-point. But it seems to me right now that we would otherwise need to provide extra metadata with the detections to make them comparable, which would change a lot more in terms of class properties and detection file contents..

Thanks, and yes, it may be the simplest way. Maybe we will end up going with that to get it fixed with an eye to a better solution later.

CJ Chamberlain, out of office

From: FelixHa notifications@github.com Sent: Saturday, February 13, 2021 9:51:40 AM To: eqcorrscan/EQcorrscan EQcorrscan@noreply.github.com Cc: Calum Chamberlain calum.chamberlain@vuw.ac.nz; Mention mention@noreply.github.com Subject: Re: [eqcorrscan/EQcorrscan] Should the self-detection time of a template be independent of other templates in the tribe? (#438)

Hi @calum-chamberlainhttps://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcalum-chamberlain&data=04%7C01%7Ccalum.chamberlain%40vuw.ac.nz%7C0c83cf68d28946fe92c308d8cf9800c8%7Ccfe63e236951427e8683bb84dcf1d20c%7C0%7C0%7C637487599098293688%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=X8j%2FPGxNhumXGFwRZYtLsaPL5ckWSeyBTdhqs%2B6dQ6Y%3D&reserved=0, thanks for your quick reply! I will create and upload the test case soon (guess on Monday), and yes I can start the PR for that where we can work on the fix. I also thought that the NaN-trace is not optimal from the overhead-point. But it seems to me right now that we would otherwise need to provide extra metadata with the detections to make them comparable, which would change a lot more in terms of class properties and detection file contents..

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Feqcorrscan%2FEQcorrscan%2Fissues%2F438%23issuecomment-778448042&data=04%7C01%7Ccalum.chamberlain%40vuw.ac.nz%7C0c83cf68d28946fe92c308d8cf9800c8%7Ccfe63e236951427e8683bb84dcf1d20c%7C0%7C0%7C637487599098303688%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=DCk3R4FSVzFgexcQ3nLC1JtIlA4lQStwF5NTZwaiQj0%3D&reserved=0, or unsubscribehttps://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FACTIM445GI5OTACMY7FL3HDS6WIFZANCNFSM4XQ5L4VQ&data=04%7C01%7Ccalum.chamberlain%40vuw.ac.nz%7C0c83cf68d28946fe92c308d8cf9800c8%7Ccfe63e236951427e8683bb84dcf1d20c%7C0%7C0%7C637487599098303688%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=IF0n5bICaFg5rGskkp5hVD6BrTLFN88yONOjgT%2FxzkE%3D&reserved=0.

eqcorrscan / EQcorrscan