Configuring librdkafka_options in the source or sink is impossible with a customConfig

[ogarcia]

I put this here more than anything so that nobody has to waste hours of their life for this, I think the fault is really in Helm and not in the template (I have opened an issue for this).

For example, having this configuration:

role: Agent
service:
    enabled: false
serviceHeadless:
    enabled: false
customConfig:
    data_dir: /vector-data-dir
    api:
        enabled: true
        address: 127.0.0.1:8686
        playground: false
    sources:
        kubernetes_logs:
            type: kubernetes_logs
    sinks:
        kafka:
            type: kafka
            inputs:
                - kubernetes_logs
            bootstrap_servers: the.kafka.server:9093
            topic: aprettytopic
            encoding:
                codec: json
            sasl:
                enabled: true
                mechanism: PLAIN
                username: user
                password: pass
            librdkafka_options:
                "security.protocol": sasl_ssl

When applied, Helm removes the quotes from "security.protocol" and leaves the string as security.protocol, which is not interpreted correctly by Vector.

I don't think there is any way to solve this problem. Not at least without it being fixed by Helm, but if anyone has any ideas, they are welcome.

[jszwedko]

One workaround Vector could do here is to flatten anything under librdkafka_options. This does seem like a potential Helm bug though, I'll respond over on that issue too.

[jszwedko]

As a workaround, you might also provide the config as JSON as suggested in the Helm issue:

librdkafka_options:
                { "security.protocol": sasl_ssl }

[ogarcia]

As described in this comment, in the end this issue is not a bug.