oxen-io / oxen-mq

Communications layer used for both the Oxen storage server and oxend
https://oxen.io
BSD 3-Clause "New" or "Revised" License
19 stars 37 forks source link

Allow injecting external tasks into lokimq job queue #7

Closed jagerman closed 4 years ago

jagerman commented 4 years ago

This allows mixing some outside task into the lokimq job queue for a category (queued up with native LMQ requests for that category) for use when there is some external process that is able to generate messages.

For example, the most immediate use for this is to allow an HTTP server to handle incoming RPC requests and, as soon as they arrive, inject them into LokiMQ's queue for the "rpc" category so that native LMQ rpc requests and HTTP rpc requests share the same thread pool and queue.

These injected jobs bypass all of LokiMQ's authentication and response mechanisms: that's up to the invoked callback itself to manage.

Injected tasks are somewhat similar to batch jobs, but unlike batch jobs the are queued and prioritized as ordinary external LokiMQ requests. (Batch jobs, in contrast, have a higher scheduling priority, no queue limits, and typically a larger available thread pool).