As a Koinos node operator, I want to limit the number of requests from jsonrpc that can be in flight at any one time.
The behavior of the jsonrpc API currently sends an MQ request for every incoming http request. This can lead to quickly saturating MQ queues and delay processing of higher priority messages.
To help mitigate this, the jsonrpc microservice should limit in flight MQ messages using a thread pool. By placing a limit at the number of requests that can be in flight at any given time, then there is an upper bound as to the number of pending requests awaiting a response from a microservice (most notably chain) allow other requests to continue to be answered. This is particularly important as requests from jsonrpc are generally regarded as the lowest priority MQ messages (their expiration does not impact usability of a node, but their inclusion can reduce performance of a node).
We need to create a thread "pool" (create multiple go routines on startup) that consume from a single channel, forwarding requests along to MQ. The http handler function needs to send requests to this channel instead of MQ.
The number of threads in the "pool" should be configurable at runtime and set to sane default value.
As a Koinos node operator, I want to limit the number of requests from jsonrpc that can be in flight at any one time.
The behavior of the jsonrpc API currently sends an MQ request for every incoming http request. This can lead to quickly saturating MQ queues and delay processing of higher priority messages.
To help mitigate this, the jsonrpc microservice should limit in flight MQ messages using a thread pool. By placing a limit at the number of requests that can be in flight at any given time, then there is an upper bound as to the number of pending requests awaiting a response from a microservice (most notably chain) allow other requests to continue to be answered. This is particularly important as requests from jsonrpc are generally regarded as the lowest priority MQ messages (their expiration does not impact usability of a node, but their inclusion can reduce performance of a node).
We need to create a thread "pool" (create multiple go routines on startup) that consume from a single channel, forwarding requests along to MQ. The http handler function needs to send requests to this channel instead of MQ.
The number of threads in the "pool" should be configurable at runtime and set to sane default value.