A difference between the precision being used by the collector and that which was being used by the validator resulted in a median value which was not verified correctly.
This anomalous system behavior was first noticed in Incident 025.
Status
Under Review
Assessment
At 0300 UTC on 8 April the regular heartbeat was missed. The issue corrected itself during the next heartbeat.
Additional Notes
Orcfax introduced an additional source to collectors earlier on in the year. With differences in programming language used for collecting and validating, the Orcfax team is seeing different levels of precision causing misalignment during validation.
Technical improvements
We are investigating:
More affective methods of determining the level of precision in the validator node.
Temporarily reducing the number of sources in collectors from six to five.
Identifying the correct retry mechanism between the validator and coop that increases publishing reliability.
Trigger
Date
2024-04-08
Summary
A difference between the precision being used by the collector and that which was being used by the validator resulted in a median value which was not verified correctly.
This anomalous system behavior was first noticed in Incident 025.
Status
Under Review
Assessment
At 0300 UTC on 8 April the regular heartbeat was missed. The issue corrected itself during the next heartbeat.
Additional Notes
Orcfax introduced an additional source to collectors earlier on in the year. With differences in programming language used for collecting and validating, the Orcfax team is seeing different levels of precision causing misalignment during validation.
Technical improvements
We are investigating:
Documentation improvements
NA