microsoft / iomt-fhir

iomt-fhir
MIT License
146 stars 68 forks source link

Prevent duplicate observations from being created #259

Closed wi-y closed 1 year ago

wi-y commented 1 year ago

In order to prevent duplicate observations from being created, we will be changing two things.

  1. Pass cancellation tokens through to downstream methods. If an event processor loses ownership of a partition we should receive a cancellation token and pass this cancellation token to our fhir import service and stop any additional requests.
  2. While cancellations are a good start, they do not necessarily handle in flight requests. So in addition, when creating an observation, we should not let the server generate the observation id. Previously, when two processors were writing to the fhir service it was possible that two creates could happen at the same time which would result in 2 observations. In this pull request the client (medtech service) will hash the id so that the id is deterministic, and only 1 observation could be created.

There are still some limitations. With option 2 you could theoretically still have a race condition. If two clients theoretically are creating an observation of the FHIR server at the same time, then two HTTP puts will occur and the 2nd HTTP put will win. This scenario should be rare because of the cancellation tokens now being checked, but I could see a case where the propagation of the cancellation token is not fast enough. If this scenario ever occurs we throw an exception.