Publish kafka messages from a thread

NSLS-II / nslsii

NSLS-II related devices

BSD 3-Clause "New" or "Revised" License

11 stars 23 forks source link

Publish kafka messages from a thread #122

Closed jklynch closed 3 years ago

jklynch commented 3 years ago

This PR modifies the function that subscribes a Kafka Publisher to the RunEngine by inserting a thread between the two. The RunEngine places documents in a queue, and the "publishing thread" takes those documents off the queue and publishes them. The intention is to insulate plans from Kafka exceptions using the same idea as is used for olog.

Significant improvements have been made to the test coverage.

mrakitin commented 3 years ago

Looks like this PR introduced the import of bluesky_kafka on the module level, overriding the conditional import implemented via https://github.com/NSLS-II/nslsii/pull/124.