Datadog Metrics are not sent when using Artillery with Fargate

[70mm1]

When trying to send metrics and events to Datadog while using Artillery with Fargate, no data is sent to Datadog. When trying the same parameters and configurations when using Artillery with the normal run command, everything works. Trying the debug environment variable behaves the same, it shows information when set for the normal run command but no information is shown when using run-fargate

Version info:

v2.0.15

Running this command:

docker run -i --rm \
                  -v .:/load-test \
                  -u root:root \
                  -e AWS_ACCESS_KEY_ID=\$AWS_ACCESS_KEY_ID \
                  -e AWS_SECRET_ACCESS_KEY=\$AWS_SECRET_ACCESS_KEY \
                  -e AWS_SESSION_TOKEN=\$AWS_SESSION_TOKEN \
                  -e DD_API_KEY=\$API_KEY \
                  -e DD_APP_KEY=\$APP_KEY \
                  -e DEBUG=plugin:publish-metrics:datadog-statsd \
                  artillery \
                  run-fargate \
                  --region eu-central-1 \
                  --subnet-ids *** \
                  --security-group-ids ***  \
                  --count 1 \
                  --cluster artilleryio-cluster \
                  --spot \
                  --output report.json \
                  /load-test/load-test.yml

I expected to see this happen: Debug messages for Datadog appearing and metrics and events being sent to Datadog.

Instead, this happened: Instead, only AWS debug messages are showing up and the test runs successfully. No events or metrics are reported to Datadog. Using the normal artillery run command works with sending events and metrics to Datadog, so I do not assume that there is an issue with the API or APP key.

Files being used: Test script

config:
  target: ***
  phases:
    - duration: 10
      arrivalRate: 1
      name: Warmup
    - duration: 20
      arrivalRate: 1
      rampTo: 10
      name: rampUp
  plugins:
    publish-metrics:
      - type: datadog
        apiKey: "{{ $env.DD_API_KEY }}"
        appKey: "{{ $env.DD_APP_KEY }}"
        apiHost: "app.datadoghq.eu"
  engines:
    playwright:
      launchOptions:
        headless: true
      aggregateByName: true
      contextOptions:
        storageState:  "{{ $dirname }}/storage.json"
  processor: "./processor.ts"

[InesNi]

Hi @70mm1 ,

Thank you for reporting this and for all the info, we will look into it and get back to you.

[InesNi]

Hi @70mm1👋🏻,

Would you mind trying to add the .env file with the DD_API_KEY, DD_APP_KEY, and DEBUG="plugin:publish-metrics:datadog-statsd to the folder where your YAML script is, and then add the --dotenv flag with its location to your command:

docker run -i --rm \
                  -v .:/load-test \
                  -u root:root \
                  -e AWS_ACCESS_KEY_ID=\$AWS_ACCESS_KEY_ID \
                  -e AWS_SECRET_ACCESS_KEY=\$AWS_SECRET_ACCESS_KEY \
                  -e AWS_SESSION_TOKEN=\$AWS_SESSION_TOKEN \
                  -e DD_API_KEY=\$API_KEY \
                  -e DD_APP_KEY=\$APP_KEY \
                  -e DEBUG=plugin:publish-metrics:datadog-statsd \
                  artillery \
                  run-fargate \
                  --region eu-central-1 \
                  --subnet-ids *** \
                  --security-group-ids ***  \
                  --count 1 \
                  --cluster artilleryio-cluster \
                  --spot \
                  --output report.json \
                  --dotenv /load-test/.env \  # just plug it in here
                  /load-test/load-test.yml

If you haven't already?

[70mm1]

Hi @InesNi, your recommended way worked 👍 However, the apiKey and appKey will be leaked in the debug logs as well set in plaintext in the container in ECS as environment variables. Is there any option to inject these Secrets over their ARN from the secrets manager so these values are used and treated as secrets as they are? Like mentioned here: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/secrets-envvar-secrets-manager.html

Thank you very much for your help!

[hassy]

Yep, Artillery can load secrets from SSM Parameter Store, with the --secret flag for the run-fargate command. You can create a parameter with type SecureString to encrypt the value (e.g. with the AWS CLI - docs). As long as it's created under /artilleryio/ path the Artillery CLI will be able to read it. These are region-specific and need to be created in the region where you run your test.

You'll need to create two parameters: /artilleryio/DD_API_KEY and /artilleryio/DD_APP_KEY, and then provide them to Artillery with:

artillery run-fargate --secret DD_API_KEY --secret DD_APP_KEY mytest.yml

Artillery will create env vars inside the running container with the same name as the parameters.

[hassy]

P.S. this uses the ECS-native mechanism here: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/secrets-envvar-ssm-paramstore.html

[70mm1]

Thank you @hassy, will try out it. By using that approach, the values are not logged when the debug options are turned on?