skatsuta commented 2 years ago

What does this PR do?

When the Datadog Lambda Extension is running, do not show error messages even if neither DD_API_KEY nor DD_KMS_API_KEY is set.

Motivation

In #101, it became clear that when the Datadog Lambda Extension is running, this library does not require either DD_API_KEY or DD_KMS_API_KEY.

Therefore, when the Extension is running, even if neither DD_API_KEY nor DD_KMS_API_KEY is set, error messages should not appear.

Testing Guidelines

Same as #101. No error messages were shown even though neither DD_API_KEY nor DD_KMS_API_KEY was set 👍

Additional Notes

Types of changes

[x] Bug fix
[ ] New feature
[ ] Breaking change
[ ] Misc (docs, refactoring, dependency upgrade, etc.)

Checklist

[x] This PR's description is comprehensive
[ ] This PR contains breaking changes that are documented in the description
[ ] This PR introduces new APIs or parameters that are documented and unlikely to change in the foreseeable future
[ ] This PR impacts documentation, and it has been updated (or a ticket has been logged)
[ ] This PR's changes are covered by the automated tests
[ ] This PR collects user input/sensitive content into Datadog

codecov-commenter commented 2 years ago

Codecov Report

Merging #102 (abb5d73) into main (23c42de) will increase coverage by 0.09%. The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #102      +/-   ##
==========================================
+ Coverage   85.21%   85.31%   +0.09%     
==========================================
  Files          13       13              
  Lines         778      783       +5     
==========================================
+ Hits          663      668       +5     
  Misses         91       91              
  Partials       24       24

Impacted Files	Coverage Δ
ddlambda.go	`78.57% <100.00%> (+0.67%)`	:arrow_up:
internal/metrics/listener.go	`84.42% <100.00%> (+0.25%)`	:arrow_up:

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

skatsuta commented 2 years ago

@tianchu Thanks for running PR checks. It seems integration tests have failed due to the lack of AWS creds: https://github.com/DataDog/datadog-lambda-go/runs/5589674292?check_suite_focus=true

Run ./run_integration_tests.sh
/home/runner/work/datadog-lambda-go/datadog-lambda-go/tests/integration_tests
No AWS credentials were found in the environment.
Note that only Datadog employees can run these integration tests.

Can you run those tests? Or is this error ignorable?

tianchu commented 2 years ago

@skatsuta Thanks for your contribution! The changes looks largely fine, I'll review and test. BTW, were you able to test and confirm the changes allowed you to use the secret arn?

skatsuta commented 2 years ago

@tianchu Yes, I have confirmed the following with my Lambda:

Only DD_API_KEY_SECRET_ARN was set and neither DD_API_KEY nor DD_KMS_API_KEY was set to the Lambda's env vars
DD Extension (Serverless Agent) was able to send traces and metrics using DD_API_KEY_SECRET_ARN
No error messages were shown from this library, which is expected by this PR's changes

Part of CloudWatch Logs of the Lambda in my test:

2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Datadog extension version : 21\|Datadog environment variables: DD_API_KEY_SECRET_ARN=***\|DD_ENV=development\|DD_LOG_LEVEL=DEBUG\|DD_SERVICE=<REDACTED>\|DD_TRACE_ENABLED=true\|DD_VERSION=<REDACTED>\|
--
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Starting daemon to receive messages from runtime...
--
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Unable to restore the state from file
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Found DD_API_KEY_SECRET_ARN value, trying to use it.
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| INFO \| Using API key set in Secrets Manager.
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Using a SyncForwarder with a 5s timeout
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| INFO \| Retry queue storage on disk is disabled
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| 'telemetry.dogstatsd.aggregator_channel_latency_buckets' is empty, falling back to default values
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| 'telemetry.dogstatsd.listeners_latency_buckets' is empty, falling back to default values
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| 'telemetry.dogstatsd.listeners_channel_latency_buckets' is empty, falling back to default values
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Forwarder started
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| dogstatsd-udp: 127.0.0.1:8125 successfully initialized
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Enabling logs collection HTTP route
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Subscribing to Logs for types: [platform function extension]
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| INFO \| dogstatsd-udp: starting to listen on 127.0.0.1:8125
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| WARN \| Error loading config: open /var/task/datadog.yaml: no such file or directory
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| INFO \| Features detected from environment:
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Setting DefaultEnv to "development" (from 'env' config option)
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Could not get hostname via gRPC: grpc client disabled via cmd_port: -1. Falling back to other methods.
LOGS    Name: datadog-agent State: Subscribed   Types: [platform,function,extension]
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| INFO \| Starting a serverless logs-agent...
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| INFO \| logs-agent started
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Adding AWS Logs collection source
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Acquired hostname from OS: "169.254.104.221". Core agent was unreachable at "/opt/datadog-agent/bin/agent/agent": fork/exec /opt/datadog-agent/bin/agent/agent: no such file or directory.
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Trace writer initialized (climit=200 qsize=78)
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Stats writer initialized (climit=20 qsize=83)
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| serverless agent ready in 109.998276ms
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| INFO \| Listening for traces at http://localhost:8126
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Starting concentrator

...(snip)...

2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Hit on the serverless.Hello route.
--
2022/03/17 13:51:27 {     "status": "debug",     "message": "datadog: Will use the Serverless Agent" }
EXTENSION   Name: datadog-agent State: Ready    Events: [INVOKE,SHUTDOWN]
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Received invocation event...
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| could not find a unique value for runtime
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| The flush strategy end has decided to not flush at moment: starting
2022/03/17 13:51:27 {     "status": "debug",     "message": "datadog: Merge X-Ray Traces is off, using trace context from Datadog only" }
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Skipping received empty payload
2022/03/17 13:51:27 Datadog Tracer v1.36.2 INFO: DATADOG TRACER CONFIGURATION {
  ...(snip)...
}
2022/03/17 13:51:27 {     "status": "debug",     "message": "datadog: Could not convert TraceContext to a SpanContext (most likely TraceContext was empty)" }

...(snip)...

2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Hit on the serverless.Flush route.
--
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Received a runtimeDone log message for the current invocation <REDACTED>
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| The flush strategy end has decided to flush at moment: stopping
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Beginning metrics flush at time 1647525087
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Received a Flush trigger
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Beginning logs flush at time 1647525087
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| INFO \| Triggering a flush in the logs-agent
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Flush in the logs-agent done.
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Finished logs flush that was started at time 1647525087
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Flushing 2 sketches to the forwarder
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Sending sketches payload : {
  ...(snip)...
}
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| The payload was not too big, returning the full payload
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Beginning traces flush at time 1647525087
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Serializing 1 tracer payloads.
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Sending trace payload : {
  ...(snip)...
}
2022-03-17 13:51:28 UTC \| DD_EXTENSION \| INFO \| Successfully posted payload to "https://6-0-0-app.agent.datadoghq.com/api/beta/sketches?api_key=********************************", the agent will only log transaction success every 500 transactions
2022-03-17 13:51:28 UTC \| DD_EXTENSION \| DEBUG \| SyncForwarder has flushed 1 transactions
2022-03-17 13:51:28 UTC \| DD_EXTENSION \| DEBUG \| Flushing 2 series to the forwarder
2022-03-17 13:51:28 UTC \| DD_EXTENSION \| DEBUG \| The payload was not too big, returning the full payload
2022-03-17 13:51:28 UTC \| DD_EXTENSION \| DEBUG \| Flushed traces to the API; time: 633.619078ms, bytes: 1669
2022-03-17 13:51:28 UTC \| DD_EXTENSION \| DEBUG \| Finished traces flush that was started at time 1647525087
2022-03-17 13:51:28 UTC \| DD_EXTENSION \| DEBUG \| SyncForwarder has flushed 1 transactions
2022-03-17 13:51:28 UTC \| DD_EXTENSION \| DEBUG \| Flushing 1 service checks to the forwarder
2022-03-17 13:51:28 UTC \| DD_EXTENSION \| DEBUG \| The payload was not too big, returning the full payload
2022-03-17 13:51:28 UTC \| DD_EXTENSION \| DEBUG \| SyncForwarder has flushed 1 transactions
2022-03-17 13:51:28 UTC \| DD_EXTENSION \| DEBUG \| Finished metrics flush that was started at time 1647525087

skatsuta commented 2 years ago

@tianchu Thank you so much for your support, I'm looking forward to the next release!

tianchu commented 2 years ago

@skatsuta Thank you for your contribution! See https://github.com/DataDog/datadog-lambda-go/releases/tag/v1.4.0

skatsuta commented 2 years ago

@tianchu Awesome! Thanks!

DataDog / datadog-lambda-go

Do not show error messages even if neither `DD_API_KEY` nor `DD_KMS_API_KEY` is set when Lambda Extension is running #102

What does this PR do?

Motivation

Testing Guidelines

Additional Notes

Types of changes

Checklist

Codecov Report