MFAshby
commented
2 years ago Additional to this; it would be really good to consider message ordering when integrating with pubsub. For our use case, it's useful to have resources created in the right order for referential integrity; i.e. our application should receive newly created Organization resources before newly created Practitioner resources, because our application code can be significantly simpler as a result (it can assume the Organization exists)
Problem I need to solve
I want to receive change notifications for data in aidbox via Google Pubsub. Using a pubsub system allows us to build decoupled systems which scale better. Using google pubsub reduces operations burden for our company as we don't have to host our own (e.g. kafka or something)
Solution I see
Configuration option in aidbox to publish change notifications for either all resources or a subset to a google pubsub topic. Change notification should contain at least the operation (create, update, delete) and the self-link of the resource (e.g. /fhir/Patient/123, or same but with the full URL)
Alternatives I've considered, but they don't work
Existing changes API allows for polling [1] but this requires additional glue code to feed that data into a messaging system.
Existing REST hooks API [2] may be configured to push to google pubsub via it's REST API [3] but I can't see an obvious way to transform to the required format for google pubsub, and authentication seems difficult to configure.
Additional context
[1] https://docs.aidbox.app/api-1/reactive-api-and-subscriptions/usdsnapshot-usdwatch-and-usdversions-api [2] https://docs.aidbox.app/api-1/reactive-api-and-subscriptions/subscriptions-1 [3] https://cloud.google.com/pubsub/docs/reference/rest/v1/projects.topics/publish