[DevOps] Determine why `vaec-cms` Lambda log messages are logged twice. - Githubissues

department-of-veterans-affairs / va.gov-team

Public resources for building on and in support of VA.gov. Visit complete Knowledge Hub:

https://depo-platform-documentation.scrollhelp.site/index.html

283 stars 205 forks source link

[DevOps] Determine why `vaec-cms` Lambda log messages are logged twice. #86844

Closed ndouglas closed 4 months ago

ndouglas commented 4 months ago

Description

Lambda log messages are being logged twice in vaec-cms and presumably will be in vaec-cie as well.

See this Slack thread.

Tasks

[x] [DevOps] Determine why log messages are being recorded twice.
[x] [DevOps] Determine which set of messages should be kept (since they have different attributes).
[x] [DevOps] Document the reason why in this thread and the decision on which to keep.
[x] [DevOps] Open up a followup ticket to address appropriately.

Acceptance Criteria

[x] Root cause has been determined and documented in this thread.
[x] Followup ticket(s) opened to address findings.

ndouglas commented 4 months ago

As suspected, this is caused by the Datadog Lambda Forwarder log subscriptions, which are superfluous when you're using the Datadog Lambda Extension. Opened department-of-veterans-affairs/checkin-devops#286 and department-of-veterans-affairs/checkin-devops#287 to address.