VictoriaMetrics / operator

Kubernetes operator for Victoria Metrics
Apache License 2.0
410 stars 141 forks source link

Missing TLS configuration during self service scrape creation by operator #1033

Open naveenbharadwaj-nc opened 2 weeks ago

naveenbharadwaj-nc commented 2 weeks ago

Hello,

VMServiceScrape object generates part of VMAgent configuration with kubernetes service discovery targets by corresponding Service. It has various options for scraping configuration of target (with basic auth,tls access, by specific port name etc.).

As per this statement from documentation, we expect that VMServiceScrapes created automatically by the operator must honor HTTPS scheme and TLSConfig when TLS is configured.

The endpoints of the automatically created VMServiceScrape for all VM components contains only Port and Path. It is from the logic here. When constructing endpoints, neither scheme nor TLSConfig is populated to VMServiceScrape. This violates the documentation.

Please fix it if its a bug. If my understanding is incorrect and there is some way to tell operator to add TLSConfig and Scheme to generated VMServiceScrapes, please provide such instructions.

f41gh7 commented 1 week ago

There is a workaround for it. Manually defined needed options at spec.serviceScrapeSpec setting.

For example:

kind: VMAlert
spec:
  serviceScrapeSpec:
    endpoints:
    - port: http
      scheme: https 

I think we should add best-effort params detection later.

naveenbharadwaj-nc commented 1 week ago

Sure. It would also mean that we have to disable selfServiceSpecCreation by setting VM_DISABLESELFSERVICESCRAPECREATION to true (just adding this information for reference). Or will it work if we dont disable? When victoriametrics-operator has an option to create ServiceScrapes automatically, its obvious that most people would rely on such an option because it is more reliable. Using such work around will force us to add a lot of code (write entire ServiceScrapes on our side for all VM components). Another downside of using work around is that if victoriametrics-operator makes any enhancements/fixes to selfServiceSpecCreation creation logic in the future, we would miss it. Its not high priority, but definitely useful if implemented on the operator side.

f41gh7 commented 1 week ago

Sure. It would also mean that we have to disable selfServiceSpecCreation by setting VM_DISABLESELFSERVICESCRAPECREATION to true (just adding this information for reference). Or will it work if we dont disable?

There is no need to disable servicescrape auto-creation. This setting allows to override default values at operator-generated servicescrape.

It's only workaround for now. Later it should be fixed and operator must respect values from extraArgs to correctly generate servicescrape.