Is your feature request related to a problem? Please describe.
Currently in ConfigWatch, we use a single thread to loop context list (consul path) to detect changes periodically (delay). And for each context (path) we use a blocking query to detect the change (wait, default to 55 seconds). This pattern will introduce huge latency of change detection. Suppose we have 10 context (path) to watch, the last one is changed but we have to wait the previous 9 contexts to finish, and that could cause 8~9 minutes latency (55*9). If we shorten the wait to 1 second or so, then we will end up with many requests in a busy loop. As we have thousands of pods, that will cause high pressure to consul, so we have to increase delay, but this causes high latency of change detection. As our business continue to grow with more pods concurrently accessing consul, we have to increase the delay until we hit the wall when the delay is too long to be acceptable by our applications.
The current design cannot have both low latency and low pressure to consul at the same time.
Describe the solution you'd like
I would like to raise a PR to change this. I want to have N threads fo N contexts, and each thread just watches a single context path, with a wait default to 8 minutes, which is long enough to make less pressure to Consul, and the change detection latency is minimized.
I think the best way to do this is to use an HTTP client that supports async Future, then we can send many requests to many contexts from a single thread, then collect the result with Future and then trigger refresh, but that means we will not use ecwid consul client, that's too much of change.
What do you think? which solution do you prefer?
Describe alternatives you've considered
N threads N contexts, or single thread with many Futures with nio/aio support.
spencergibb
commented
1 year ago
Is your feature request related to a problem? Please describe. Currently in ConfigWatch, we use a single thread to loop context list (consul path) to detect changes periodically (delay). And for each context (path) we use a blocking query to detect the change (wait, default to 55 seconds). This pattern will introduce huge latency of change detection. Suppose we have 10 context (path) to watch, the last one is changed but we have to wait the previous 9 contexts to finish, and that could cause 8~9 minutes latency (55*9). If we shorten the wait to 1 second or so, then we will end up with many requests in a busy loop. As we have thousands of pods, that will cause high pressure to consul, so we have to increase delay, but this causes high latency of change detection. As our business continue to grow with more pods concurrently accessing consul, we have to increase the delay until we hit the wall when the delay is too long to be acceptable by our applications.
The current design cannot have both low latency and low pressure to consul at the same time.
Describe the solution you'd like I would like to raise a PR to change this. I want to have N threads fo N contexts, and each thread just watches a single context path, with a wait default to 8 minutes, which is long enough to make less pressure to Consul, and the change detection latency is minimized.
I think the best way to do this is to use an HTTP client that supports async Future, then we can send many requests to many contexts from a single thread, then collect the result with Future and then trigger refresh, but that means we will not use ecwid consul client, that's too much of change.
What do you think? which solution do you prefer?
Describe alternatives you've considered N threads N contexts, or single thread with many Futures with nio/aio support.
Additional context N/A