Calculate "Measure of effectiveness" of detector algorithms

[IanMayo]

For the first dataset, an analyst has reviewed the data, and determined when the turns start/stopped.

data1_turns.txt

We should include this against the first dataset in the RawData spreadsheet. So, I think we need to introduce two new columns (D & E). If we exchange Start and End with our leg counter (1, 1.5, 2, 2.5, etc) then we can match predicted turns against the truth data.

The following overall measures of effectiveness are to be considered:

• Manoeuvres that are not detected at all
• Manoeuvres that are detected late (and how late)
• Manoeuvre whose end is detected early (and how early)

Plus these for each manoeuvre:

• Average time before manoeuvre it is detected
• Average time after manoeuvre the end is detected

I think we need the measures of effectiveness against each turn detector (Delta and Delta 2).

Hmm, let's talk through how to present the measures of effectiveness. I think we don't need them to be in the data table on the Analysis worksheet, since we don't need all of the time values. I may be mistaken, but I think we could have one row per turn. Aah, then we can have one column for the Leg id, one for the analyst time, one for Delta time, then Delta 2 time, then the measures of effectiveness (one per column).

[IanMayo]

Have made a start at this, here: https://github.com/debrief/mda/commit/94ca58d6048445506f05491f1a495fb0f604776c

[GDXFR]

Regarding the comparison performance, we can see that:

• Neither delta1 nor delta2 detect the manoeuvre at 17241 sec with trigger set to 5
• Delta2 detects it with trigger set to 4
• Delta2 gives false positive with trigger set to 3deg

[IanMayo]

Ok, Iain has implemented this.