vectordotdev / helm-charts

Helm charts for Vector.
https://vector.dev
Mozilla Public License 2.0
102 stars 89 forks source link

Opentelemetry sources define 2 addresses: these aren't exposed by the resultant service #370

Open grahamnicholls opened 4 months ago

grahamnicholls commented 4 months ago

Apologies for terminology mistakes - I am pretty new to k8s. If an opentelemetry source is configured, it requires 2 address values; one for grpc and one for http (as in the config extract below). However, when helm renders the template, no endpoints are created for the opentelemetry source.

sources:
                vector_logs:
                   type: internal_logs
                vector_metrics:
                   type: internal_metrics
                   scrape_interval_secs: 15
                   namespace: vector

                otlp:
                   type: opentelemetry
                   grpc:
                      address: 0.0.0.0:4317
                     tls:
                       enabled: false
                   http:
                     address: 0.0.0.0:4318
                     tls:
                       enabled: false

                prometheus:
                   type: prometheus_remote_write
                   address: 0.0.0.0:9090

Produces a service looking like so:

$ kd -n gnotel svc vector-gnotel

Name:              vector-gnotel
Namespace:         gnotel
Labels:            [app.kubernetes.io/component=Aggregator](http://app.kubernetes.io/component=Aggregator)
                   [app.kubernetes.io/instance=vector-gnotel](http://app.kubernetes.io/instance=vector-gnotel)
                   [app.kubernetes.io/managed-by=Helm](http://app.kubernetes.io/managed-by=Helm)
                   [app.kubernetes.io/name=vector](http://app.kubernetes.io/name=vector)
                   [app.kubernetes.io/version=0.33.1-distroless-libc](http://app.kubernetes.io/version=0.33.1-distroless-libc)
                   [helm.sh/chart=vector-0.27.0](http://helm.sh/chart=vector-0.27.0)
Annotations:       [argocd.argoproj.io/tracking-id](http://argocd.argoproj.io/tracking-id): gnotel:/Service:gnotel/vector-gnotel
Selector:          [app.kubernetes.io/component=Aggregator,app.kubernetes.io/instance=vector-gnotel,app.kubernetes.io/name=vector](http://app.kubernetes.io/component=Aggregator,app.kubernetes.io/instance=vector-gnotel,app.kubernetes.io/name=vector)

Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                 172.20.25.132
IPs:                172.20.25.132
Port:              api  8686/TCP
TargetPort:        8686/TCP
Endpoints:         [100.71.26.217:8686](http://100.71.26.217:8686/),[100.71.26.222:8686](http://100.71.26.222:8686/)
Port:              prometheus  9090/TCP
TargetPort:        9090/TCP
Endpoints:         [100.71.26.217:9090](http://100.71.26.217:9090/),[100.71.26.222:9090](http://100.71.26.222:9090/)

Session Affinity:  None
Events:            <none>

As can be seen, endpoints are created for the API, and for the prometheus source, but not for the opentelemetry source.

jszwedko commented 4 months ago

Thanks for filing this @grahamnicholls ! This looks to be a bug in the Helm chart templating. It only looks for address fields at the top-level and for the opentelemetry source they are nested. The fix for this would need to be somewhere around: https://github.com/vectordotdev/helm-charts/blob/64ea7a50c64fad3a520643f46be5209f74b71c6c/charts/vector/templates/_helpers.tpl#L92-L110

In absence of that, you can configure the ports directly via the containerPorts and service.ports. Note I think this will override the automatic generation so all needed ports will need to be specified.