refraction-networking / conjure

Conjure Refraction Networking station code
https://refraction.network
Apache License 2.0
66 stars 19 forks source link

Registration ingest ToC-ToU race wasting resources #250

Closed jmwample closed 8 months ago

jmwample commented 8 months ago

In the registration ingest pipeline we check if the incoming registration has been seen before, as it is possible with certain registrars that more than one registration are seen for a single connection.

In order to do so we track all seen registrations in the registration manager and check against the state when a new registration is received.


Issue: By necessity the ingest pipeline is multi-threaded. This means that two registrations can be in the ingest pipeline simultaneously. The Time-of-Check to Time-of-Use error happens because there the step that checks whether a registration has been seen before and the step that adds the registration to the registration managers state are independent. This means that registrations being ingested in parallel could both pass the RegistrationExists step and continue through to the end of the pipeline. There isn't any issue with correctness when this happens, but we waste a liveness test and a redis message.

Fix: This PR combines the two steps (check for existence and tracking the incoming registration) into one step protected by the registration manager's internal mutex lock. So it is no longer possible to have more than one registration per connection transit the entire ingest pipeline, even if they show up at the same time in separate threads.