clemire
commented
1 week ago I deprioritized this because:
- this issue should not cause the entitlement check to fail to complete, and in fact only happens as described if the check did complete
- we solved the outage that was motivating this investigation. I think this only makes sense to look into at this time if our xchain stack fails to produce a check result.
Figment logs indicate that nodes sometimes do not respond because they may be lagging behind other nodes, to the point that the entitlement was already processed by a quorum of nodes when a lagging node is starting. This change would add logging to the chain monitor to detect when an event is processed > 4s after the block occurred, as a way to determine how far behind the entitlement request a node is when it (finally) responds.