v1beta - minimumStrokeQuality settings is possibly ignored

[Abasz]

I noticed (and already mentioned in another discussion) that for certain cases the minimumStrokeQuality settings seems to be ignored, yet this results in the best possible stroke detection.

https://github.com/laberning/openrowingmonitor/discussions/122#discussioncomment-4849548

Also I had a session on C2 and recorded it and tweaked the settings. This is the raw csv

C2-12022023-interval.csv

Below are the settings:

  rowerSettings: {
    ...rowerProfiles.Concept2_RowErg,
    flywheelInertia: 0.1001,
    maximumTimeBetweenImpulses: 0.70,
    sprocketRadius: 1.5,
    flankLength: 15,
    minimumStrokeQuality: 1,
    minumumRecoverySlope: 0,
    autoAdjustRecoverySlope: false,
    autoAdjustRecoverySlopeMargin: 0.035,
    minumumForceBeforeStroke: 0,
    minimumDriveTime: 0.2,
    minimumRecoveryTime: 0.2,
    dragFactor: 110,
    autoAdjustDragFactor: true,
    minimumDragQuality: 0.96,
    dragFactorSmoothing: 1
  }

With the above settings I get 710 strokes which is actually the perfect number:

My expectation would be to have zero strokes with such a high quality number as the docs state:

//Setting this too high will stop
    // the recovery phase from being detected.

Note if I decrease the flankLength, the lower the flankLength the less stroke is detected. Increasing does not make a difference.

[JaapvanEkris]

Why a flanklength of 15? That is 2,5 rotations of the flywheel and thus unstable. You need to use 12.

[JaapvanEkris]

Also I had a session on C2 and recorded it and tweaked the settings. This is the raw csv

C2-12022023-interval.csv

Are you absolutely sure this originates from a good ORM install? When I run my ORM install on your data, total chaos follows, so something is wrong with your install, as is shown with the raw file. Our raw files include more precise data, which is critical for ORM to function.

To compare, a snippit from your file:

0.012055
0.01203
0.012062
0.012212
0.012302
0.012199
0.012118
0.012092
0.012178
0.012312
0.012378
0.012264
0.012196
0.012194
0.012264
0.01238
0.012455
0.012354
0.012296
0.012264
0.012328
0.012474
0.012547
0.012455
0.012376
0.012331
0.012392
0.012522
0.012578
0.012421
0.012313
0.012243

When compared to our raw files, you can clearly see a difference:

0.009753000000046086
0.009763000000020838
0.009865999999988162
0.009972999999945387
0.010043999999993503
0.009861000000000786
0.009792000000061307
0.009851999999909822
0.009952999999995882
0.010056000000076892
0.010100999999963278
0.009915999999975611
0.009841000000051281
0.009904000000005908
0.009987000000023727
0.010092999999983476
0.010157999999933054
0.009989000000018677
0.009918999999968037
0.009984000000031301
0.010098999999968328
0.010153000000059365

I checked our automated tests on the GoF calculation, and they explicitly tests for cases where the GoF should be below 1, so that code is working. In the code you see that in the transition from drive to recovery, the GoF is explicitly checked.

Attached you'll find three RowingData files of the same row, one with minimumStrokeQuality 1, one with minimumStrokeQuality 0.95 and the last with minimumStrokeQuality of 0.01. The last is total chaos. Proving that minimumStrokeQuality is absolutely NOT IGNORED. It might not work as you expect, but it definitely is working.

See C2_RowErg_2023-02-13_1013_Rowing_6K_Drag_93_minimumStrokeQuality_1.00.csv C2_RowErg_2023-02-13_1015_Rowing_6K_Drag_93_minimumStrokeQuality_0.01.csv C2_RowErg_2023-02-13_1010_Rowing_6K_Drag_93_minimumStrokeQuality_0.95.csv

BTW, this is my current config:

  rowerSettings: {
    numOfImpulsesPerRevolution: 6,
    sprocketRadius: 1.4,
    maximumStrokeTimeBeforePause: 6.0,
    dragFactor: 110,
    autoAdjustDragFactor: true,
    minimumDragQuality: 0.95,
    dragFactorSmoothing: 3,
    minimumTimeBetweenImpulses: 0.005,
    maximumTimeBetweenImpulses: 0.023,
    flankLength: 12,
    smoothing: 1,
    minimumStrokeQuality: 0.95,
    minumumForceBeforeStroke: 0,
    minumumRecoverySlope: 0.00070,
    autoAdjustRecoverySlope: true,
    autoAdjustRecoverySlopeMargin: 0.04,
    minimumDriveTime: 0.40,
    minimumRecoveryTime: 0.90,
    flywheelInertia: 0.10164,
    magicConstant: 2.8
    }

[Abasz]

You are right about the fact that the raw csv was not recorded by ORM but with the MCU (having the Rpi in the local Gym is rather cumbersome, the ESP32 with a battery is much more suitable for this purpose).

But why would that matter for ORM in terms of the issue with the mimimumStrokeQuality? Basically the resolution is slightly less but I dont see how its possible that with a mimimumStrokeQualty of 1 (perfect) any stroke is detected at all. It should be regardless of the currentDt resolution.

With 12 I was not able to tweak the strokes. With 12 (and minimumStrokeQuality: 0.506), the best I could achieve was 711. I chased down this and there was a double detection around stroke 582 that did not occur with 15 so I considered I better.

(actually with 0.5082 I get 710, but it still duplicates the stroke at 581 and 582 while misses a stroke between 541 and 542)

Besides why is 15 unstable? there should be way more 15 data-points (i.e. 2.5 rotations) per a recovery. Especially if the minimumRecoveryTime is set to something like 0.9.

[Abasz]

With your settings I get 290 strokes. But I think this is ok as the machines I used are not that well maintained in the Gym. The one I used even wobbled at cartain speeds quite heavily. So I would not be surprised that your setting does not work for me out of the box.

[Abasz]

Attached you'll find three RowingData files of the same row, one with minimumStrokeQuality 1, one with minimumStrokeQuality 0.95 and the last with minimumStrokeQuality of 0.01. The last is total chaos. Proving that minimumStrokeQuality is absolutely NOT IGNORED. It might not work as you expect, but it definitely is working.

See C2_RowErg_2023-02-13_1013_Rowing_6K_Drag_93_minimumStrokeQuality_1.00.csv C2_RowErg_2023-02-13_1015_Rowing_6K_Drag_93_minimumStrokeQuality_0.01.csv C2_RowErg_2023-02-13_1010_Rowing_6K_Drag_93_minimumStrokeQuality_0.95.csv

I am somewhat confused on this, as there are strokes detected with minimumStrokeQuality 1 (13_1013_Rowing_6K_Drag_93_minimumStrokeQuality_1.00) even in your file. It detected 588 strokes. As I said my expectation was/is (but may be I am misunderstanding this setting based on the docs) that with a minimumStrokeQuality of 1 no recovery should be detected as it is too perfect of a goodness of fit for data generated by rowing machines.

[JaapvanEkris]

But why would that matter for ORM in terms of the issue with the mimimumStrokeQuality? Basically the resolution is slightly less but I dont see how its possible that with a mimimumStrokeQualty of 1 (perfect) any stroke is detected at all. It should be regardless of the currentDt resolution.

It might not matter, but it also could. The raw file you provided showed a very familiar pattern (a bit sinoid with a period of 6 measurements), which suggests that the same systematic error I found in the two flywheels I've tested, is also present in your gym's flywheel. That shouldn't be the problem.

But trimming the number also removes a source of random noise. That is an issue, as the regression algorithm used (OLS) assumes a random error, not a systematic one, and OLS isn't a robust algorithm (i.e. it handles violating its assumptions badly). So removing randomness from the series might actually hurt the algorithm's performance. In the development of the current engine, I've seen this before: when applied to noise filtered currentDt data (i.e. obsious errors corrected by high-low pass filters), the Goodness of Fit will be unnaturally high because of it, while the data isn't too good.

With 12 I was not able to tweak the strokes. With 12 (and minimumStrokeQuality: 0.506), the best I could achieve was 711. I chased down this and there was a double detection around stroke 582 that did not occur with 15 so I considered I better.

(actually with 0.5082 I get 710, but it still duplicates the stroke at 581 and 582 while misses a stroke between 541 and 542)

Interesting. Please note that minimumStrokeQuality isn't the only mechanism present. minimumStrokeQuality prevents the stroke detection mechanism to go too early into a Recovery phase due to a specific slope being found in the currentDt. minimumStrokeQuality safeguards that specific issue. But for detecting a Drive, there it isn't present (as in a Drive phase, currentDt's aren't supposed to be in a straight line as the acceleration is far from constant). Autoadjusting the slope, or using minimumDriveTime and minimumRecoveryTime to prevent switching phase while not intended, might help here as well.

Besides why is 15 unstable? there should be way more 15 data-points (i.e. 2.5 rotations) per a recovery. Especially if the minimumRecoveryTime is set to something like 0.9.

There is a systematic error in your (and mine) data from a C2: the magnets are placed as such that they create a sinoid with period 6, where the times are systematically too short for one series of magnets and too long for others. I assumed that using two full rotations would sufficiently reduce specific systematic noise.

When you look at the behaviour of a pure sinoid with linear regression, you see the following:

It could work, and might even be needed if the machine is indeed badly maintained. But a side-effect of a long flanklength is that it tends to kill much of the details, and thus information like the force curve etc..

[JaapvanEkris]

Attached you'll find three RowingData files of the same row, one with minimumStrokeQuality 1, one with minimumStrokeQuality 0.95 and the last with minimumStrokeQuality of 0.01. The last is total chaos. Proving that minimumStrokeQuality is absolutely NOT IGNORED. It might not work as you expect, but it definitely is working.

See C2_RowErg_2023-02-13_1013_Rowing_6K_Drag_93_minimumStrokeQuality_1.00.csv C2_RowErg_2023-02-13_1015_Rowing_6K_Drag_93_minimumStrokeQuality_0.01.csv C2_RowErg_2023-02-13_1010_Rowing_6K_Drag_93_minimumStrokeQuality_0.95.csv

I am somewhat confused on this, as there are strokes detected with minimumStrokeQuality 1 (13_1013_Rowing_6K_Drag_93_minimumStrokeQuality_1.00) even in your file. It detected 588 strokes. As I said my expectation was/is (but may be I am misunderstanding this setting based on the docs) that with a minimumStrokeQuality of 1 no recovery should be detected as it is too perfect of a goodness of fit for data generated by rowing machines.

It could kill all strokes on a mechanically weaker machine, as reaching 1 isn't possible.

Please note that minimumStrokeQuality isn't intended to be the one and only source for stroke detection. There are several, where minimumStrokeQuality is used to delay the slope based recovery detection until you are certain the downward slope is a credible decelleration (i.e. not too much noise in the stroke). Stoke detection also looks at the torque present/absent via the minumumForceBeforeStroke parameter. In a nutshell:

• a drive is detected when the torque is above a threshold AND the slope of the currentDt's is below the minumumRecoverySlope (i.e. accelerating), AND the minimumRecoveryTime is exceeded
• a recovery is detected when the there is no torque present OR (the slope of the currentDt's is consistent with the minumumRecoverySlope AND the GoF exceeds the minimumStrokeQuality), AND the minimumDriveTime is exceeded.

So at some point, you might only be switching on the torque and minimumDriveTime. For datarich machines with high quality instantanous metrics, this might work well. But as torque can be chaotic (it is an instantaneous measurement) and for some machines even practically unusable due to a total lack of measurement points, you should aim in balancing it as such that either are likely to enable the transition to the recovery when possible. For reliability of the drive-time (and thus dependent metrics), this is quite essential. By setting minimumStrokeQuality to 1 you are essentially disabling one of the safeguards for reliable stroke detection.

[Abasz]

Hi, thanks for the clarifications. Last night I drilled into the code very deep and did some testing and charting of the slopes and now I understand the different levels of the stroke detections settings better.

The docs were slightly miss leading in this respect as basically the mimimumStrokeQualty: 1 essentially switches off the slope based recovery detection (so basically enables solely torque based detection).

I reread the docs as well as the physics explanation and I think that part is slightly silent about the fact that actual ORM stroke detection uses 2 main levels (and one of these having two sublyers) all of which has different thresholds. The two main is torque and slope, while slope has a pure acceleration/deceleration or an adjustable minimum.

Actually now that I fully understand these levels it all makes perfect sense. Also I was able to debug the issue (which is indeed not that minimumStrokeQualty is ignored), but rather the following:

if I use flankLength of 12 with my data for some reason as a consequence of the median filter/averaging (if any) the negative torques disappear (I tried to have a gridline at 0). Have a look at the below chart that shows the slopes on one axis and torque on the other:

After around stroke 11-12 the torque does not go to the negative range, it remains positive....

Now because of this the torque based detection does not kick in and since with minimumStrokeQualty: 1 I switched off the the slope based detection strokes get missed.

When I increase the flankLength to 15, the torque values are better in terms of showing negative amounts.

Due to the above this performs better in terms detecting the stroke.

The issue is indeed the minor, systematic inaccuracy of the magnet and hence the timing trigger:

There is a systematic error in your (and mine) data from a C2: the magnets are placed as such that they create a sinoid with period 6, where the times are systematically too short for one series of magnets and too long for others. I assumed that using two full rotations would sufficiently reduce specific systematic noise.

One thought on this: would the assumption that for such type of systematic errors a simple averaging (or no filtering at all) results in better shaped curve, hold true? I mean the median filter removes the outliers but here that is not what is really needed, we want to keep all the data points as the deviation is not noise but rather a systematic pattern. My question is based on the theory that the median filter changes the shape of the curve due to removing the outliers instead of reducing the deviation.

I did quick excel plot and it shows that the median gives a steeper slope compared to the raw and the avg method (avg being in the middle). I wonder what are your thoughts on this.

[JaapvanEkris]

Hi, thanks for the clarifications. Last night I drilled into the code very deep and did some testing and charting of the slopes and now I understand the different levels of the stroke detections settings better.

The docs were slightly miss leading in this respect as basically the mimimumStrokeQualty: 1 essentially switches off the slope based recovery detection (so basically enables solely torque based detection).

I reread the docs as well as the physics explanation and I think that part is slightly silent about the fact that actual ORM stroke detection uses 2 main levels (and one of these having two sublyers) all of which has different thresholds. The two main is torque and slope, while slope has a pure acceleration/deceleration or an adjustable minimum.

I updated the documentation this morning, see https://github.com/JaapvanEkris/openrowingmonitor/blob/v1beta_updates/docs/rower_settings.md#setting-flanklength-and-minimumstrokequality

When I increase the flankLength to 15, the torque values are better in terms of showing negative amounts.

If it works, don't touch it :). You could also increase the minumumForceBeforeStroke, which drives the number where the torque is cut off.

There is a systematic error in your (and mine) data from a C2: the magnets are placed as such that they create a sinoid with period 6, where the times are systematically too short for one series of magnets and too long for others. I assumed that using two full rotations would sufficiently reduce specific systematic noise.

One thought on this: would the assumption that for such type of systematic errors a simple averaging (or no filtering at all) results in better shaped curve, hold true? I mean the median filter removes the outliers but here that is not what is really needed, we want to keep all the data points as the deviation is not noise but rather a systematic pattern. My question is based on the theory that the median filter changes the shape of the curve due to removing the outliers instead of reducing the deviation.

I've played with it. The running median filter is still in there (i.e. the smoothing filter), and I've experimented with it. I've played with values between 1 and 6, without much success. My experience is that when you activate the filter, I ended up with a lot of noise and missed strokes. But I kept all other settings identical to the current setup. I found that the force curve etc. became more chaotic. For me, the regression algorithms were much more effective in dealing with the noise on their own when compared to the median filter followed by the regression algorithm. But your results may differ.

[Abasz]

e played with it. The running median filter is still in there (i.e. the smoothing filter), and I've experimented with it. I've played with values between 1 and 6, without much success. My experience is that when you activate the filter, I ended up with a lot of noise and missed strokes. But I kept all other settings identical t

I dont want to reinvent the wheel it was just a thought. I run a few limited test as well ad it did not make a big difference that is why I asked (before spending more time on this :).

The only thing is that I am quite annoyed that I do not understand the reason for not getting enough negative torques... in theory it should work beautifully. I mean on the recovery there could really have no positive force on an average :) I run the same test with flankLength of 5, now that is very noisy but I get more negative values (and a lot higher positive ones as well of course).

one more question that popped in my mind when looking these graphs: They show that the at the recovery both the slope values and the torque values (regardless whether they are positive or not) resemble to flat lines. I wonder if using regression on these to determine a very flat slope get a good stroke detection. What I mean is that if the torque within the flank fitted with a linear regression (as condition one) and the related slope (with a set GoF) is within a small range AND the deltaTimes slope is also positive, we conclude that it must be recovery (instead of looking for a specific torque threshold).

If it works, don't touch it :). You could also increase the minumumForceBeforeStroke, which drives the number where the torque is cut off.

Actually this is the setting that should be increased for my case. I did not understood the importance of this and also the range. But now I have a lot of torque values and I see what would be an appropriate number.