consider async operation within sr3

[petersilva]

in v03_wip, there are already fewer timing loops thatn in v2, but we could get down to 1 loop for the entire thing. One benefit would be code that is much easier to follow because we have partial async now, but not entirely. Another benefit would be that analysts have often wanted to name multiple upstream sources for a single configuration, but current loop can only accomodate one easily. quick peeks at each source that return data only when it is available would better match how the app works.

• True Async: Single timing loop for entire app. Requires a retry_queue for posting.

  • requires a place to store failed/deferred posts (a second retry queue.)
  • changing events to have "after_post" processing of to append to outbound retry queue.
  • Do we have logic, like outbound winnow, where as long as published once, ok?
  • MQTT reception is already async, just need to fix transmit.

• Multi-source (requires async):

  • allow >1 broker and post_broker to be specified, gather from all brokers, post to all brokers. effect of true async.

[petersilva]

worklists might need to change... and we might need multiple ones for posting... if posting fails, we currently loop... in async model, would need to create a queue, and try again later. so might need a worklist of posted

[petersilva]

@reidsunderland did the post retry queueing for #466 ... at some point we should have a look at this and consider implementing the rest.

[petersilva]

https://github.com/MetPX/sarracenia/pull/1295 - completed implementation of async for consuming from upstream.

[petersilva]

so, I guess this is done?