search

VictoriaMetrics / operator

Kubernetes operator for Victoria Metrics
Apache License 2.0
410 stars 141 forks source link

Is there a way to control the parameters of the config-reloader container in a VMAgent instance? #911

Closed hansedong closed 3 months ago

hansedong commented 4 months ago

I have deployed VMAgent, and every time VMStaticScrape changes, the collection configuration of VMAgent needs to wait for a period of time to take effect. The waiting time is 3 minutes.

The following is the log of VMAgent.

level=info ts=2024-03-26T09:03:06.147844519Z caller=reloader.go:376 msg="Reload triggered" cfg_in=/etc/vmagent/config/vmagent.yaml.gz cfg_out=/etc/vmagent/config_out/vmagent.env.yaml watched_dirs="/etc/vm/relabeling, /etc/vm/stream-aggr"
level=info ts=2024-03-26T09:06:06.211457525Z caller=reloader.go:376 msg="Reload triggered" cfg_in=/etc/vmagent/config/vmagent.yaml.gz cfg_out=/etc/vmagent/config_out/vmagent.env.yaml watched_dirs="/etc/vm/relabeling, /etc/vm/stream-aggr"

I see that the configuration item watch-interval is not defined in the container config-reloader currently started in VMAgent. The default value for this parameter is 3m. I would like to adjust this parameter, is there any way to do it?

Haleygo commented 4 months ago

Unfortunately the --watch-interval=3m0s in default prometheus config-reloader is not supported to be changed. You can use the custom config-reload image victoriametrics/operator:config-reloader-v0.38.0 by adding env VM_USECUSTOMCONFIGRELOADER: true in operator, this config-reloader watches related secret(mounted as config file) instead of file itself, and reload the process once change being made.

f41gh7 commented 3 months ago

Since v0.43.0 operator adds new field configReloaderExtraArgs for VMAgent, VMAlert, VMAuth and VMAlertmanager CRDs. It allows to configure params for custom config-reloader container, provided by operator.

In any other cases, it's possible to use container patch:

kind: VMagent
spec:
 containers:
 - name: config-reloader
    args:
    # note, you must provide a full list of args, not only your changes.
    - change-any-args