[Bug]: Vault Reference not being permitted for trace_agent_url

[mfischler]

Kong Version

3.5.0

Plugin Version

0.1.2-1

On which environment your Kong instance is running?

Kubernetes

Plugin Configuration

env:
  datadog_agent_host:
      valueFrom:
        fieldRef:
          fieldPath: status.hostIP
  datadog_agent_tracing_url: "http://$(KONG_DATADOG_AGENT_HOST):8126"

- name: ddtrace
  config:
    agent_host: "{vault://env-vault/KONG_DATADOG_AGENT_HOST}"
    trace_agent_url: "{vault://env-vault/KONG_DATADOG_AGENT_TRACING_URL}"
    service_name: my-service
    environment: "{vault://env-vault/MYSERVICE_ENVIRONMENT}"
    version: "{vault://env-vault/MYSERVICE_VERSION}"
    header_tags:
      - header: X-Gateway
        tag: gateway-name

What happened?

When using Kong Vault references, the DDTrace plugin fails to start providing the error:

2024/03/27 20:59:23 [error] 1#0: init_by_lua error: /usr/local/share/lua/5.1/kong/init.lua:718: error parsing declarative config file /kong_dbless/kong.yml:in 'plugins': - in entry 3 of 'plugins': in 'config': in 'trace_agent_url': missing host in urlstack traceback: [C]: in function 'error' /usr/local/share/lua/5.1/kong/init.lua:718: in function 'init' init_by_lua(nginx-kong.conf:45):3: in main chunknginx: [error] init_by_lua error: /usr/local/share/lua/5.1/kong/init.lua:718: error parsing declarative config file /kong_dbless/kong.yml:in 'plugins': - in entry 3 of 'plugins': in 'config': in 'trace_agent_url': missing host in urlstack traceback: [C]: in function 'error' /usr/local/share/lua/5.1/kong/init.lua:718: in function 'init'

[dmehala]

Hi @mfischler,

It seems you are using the main branch. #36 removed referenceable fields, resulting in Kong Vault values not resolving them anymore. This decision might seems confusing at first glance but if Kong fails to resolve ${vault://env/A}, it silently remains unresolved and potentially going unnoticed.

Nevertheless, I've introduced support for Datadog environment variables (DD_ENV, DD_VERSION, DD_SERVICE, etc.). The recommended approach is to define these environment variables and let the plugin handle the resolution.

If I am not wrong, the upper part is from an Helm chart? If yes, then you should use customEnv as env are prefixed with KONG_.

customEnv:
  DD_AGENT_HOST:
      valueFrom:
        fieldRef:
          fieldPath: status.hostIP
  DD_ENV: $(MYSERVICE_VERSION)
  DD_VERSION: $(MYSERVICE_VERSION)

- name: ddtrace
  config:
    service_name: my-service
    header_tags:
      - header: X-Gateway
        tag: gateway-name

Let me know if it solve your issue.

EDIT: By the way, you were the one who requested support for Kong Secrets Manager (#24). However, there doesn't seem to be anything secretive about the service name, environment, or version. Did you envision a specific use case for this feature, or was a workaround to dynamically define those fields with environment variables?

[mfischler]

Able to modify pipeline to inject DD Environment variables instead of Kong vault references. Seems to be working without issues. Tested Custom Headers and able to confirm that it is working as expected.

Request for using Kong Secrets Manager (#24) appears to be satisfied with using environment variables instead. Had a business case where we wanted to restrict changes with Kong Gateway deployment but can use the environment variables instead.

[dmehala]

Great to hear it's working well for you! Thanks for trying out our recent changes. If you need anything else, just let us know.