search

kiwigrid / k8s-sidecar

This is a docker container intended to run inside a kubernetes cluster to collect config maps with a specified label and store the included files in a local folder.
MIT License
613 stars 183 forks source link

#minor Add an option to send request only once per batch #356

Open ahtohio opened 3 months ago

ahtohio commented 3 months ago

Reloading on each received event (with watch method) can be quite costly. I run Grafana provisioning and asking it to reload all my provisioned configmaps (>300) on each event is not needed and only increases cpu load.

Reload once after each WATCH_SERVER_TIMEOUT seconds should be enough.

I've added an option REQ_ONCE_PER_BATCH that can toggle this reload-once-per-watch-cycle reload behaviour, false by default to not interfere with existing logic.

ChristianGeie commented 2 months ago

Hi @ahtohio, thx for contributing. I like your reload idea to save cpu time. Changes looks good to me so far.

Do you have any ideas about whether the function can be included into the tests?

ahtohio commented 2 months ago

Hi, @ChristianGeie I've added some tests related to this functionality. For some reason, at the last commit all jobs failed for quite unknown reasons, but new workflow on the same commit in my personal branch worked well. But I can't restart/create new workflow in this repo, it seems

ChristianGeie commented 1 month ago

thx @ahtohio . I do a lot of changes regarding workflow actions in the last couple of weeks, please excuse that sometimes the tests didn't work reliably. Otherwise your changes look pretty good to me.

ahtohio commented 1 month ago

Hi, @ChristianGeie, it seems all's good, only approval left

isubasinghe commented 1 week ago

@ChristianGeie this is a pretty important fix in my opinion, my 12 core CPU gets hammered while running our application.

If there is anything I can do to progress this PR, please let me know.