SPIKE: Remove workaround from VA Profile integration lambda

[kalbfled]

• [ ] Discuss with QA

Timeboxed to 1/2 day! If the resolution still is unknown let's sync and discuss strategy/priority.

Describe the bug

The VA Profile opt-in/out lambda contains code that populate the SSL environment with all the CA certificates in a given folder using the "capath" parameter to ssl.create_default_context. As the code comments state, nothing is being loaded using the approach.

To workaround the problem, we are instead loading two certificates directly using the "cafile" parameter instead. This works, but the specific CA certificates being loaded eventually will need to change again when VA Profile updates their server certificate. The best solution would be a working revision of the "capath" approach.

Possible Solution

Please read the issue description and comments for vanotify-team 518. The CA certificates are made available to the lambda as part of a layer, and it's possible that the lambda didn't actually update in AWS, as was the case with with the .pem file update.

• [ ] Download the layer "VA_CAs" from AWS Lambda console, and compare this .zip file's content with the contents of the VA_CAs.zip file found here.

If the .zip files' contents match, this "possible solution" is not going to work. Otherwise, execute these steps:

• [ ] In Lambda console, manually create a new version of the VA_CAs layer by uploading the .zip file from the vanotify-infra repo and, also in the console, set the Profile opt-in-out lambda to use that new version.
• [ ] On a new branch, modify lambda_functions/va_profile/va_profile_opt_in_out_lambda.py by uncommenting line 125 and deleting lines 126-128.
• [ ] Deploy the Profile integration lambda to Dev, and verify it can process a POST request with a 200 response.

Expected behavior

The lambda execution environment has access to all CA certificates in the "VA_CAs" layer using "capath".

Impact

The current workaround means that the next CA chain change will require code changes again. Without the workaround, only the lambda layer will need to update.

Validation

• [ ] Make a POST request from VA Profile to the caching endpoint. You can use the AWS Lambda console "test" tab to do this. The logs should show Notify returning a 200 response to Profile, Notify making a PUT request to Profile, and Profile responding to the PUT request with a 200. Anything else is erroneous.

[mjones-oddball]

Hey team! Please add your planning poker estimate with Zenhub @cris-oddball @EvanParish @ianperera @jakehova @kalbfled @ldraney

[Eallan919]

@kalbfled please add acceptance criteria

[kalbfled]

This might be related to the recent problem wherein we update the certs lambda layer, successfully deployed it with Terraform, and later discovered that the layer wasn't actually updated in AWS. That issue has been resolved. Maybe the workaround isn't necessary anymore.

[tabinda-syed]

@k-macmillan @mjones-oddball Please see Dave's note above. Sounds like this ticket is no longer needed?

[mjones-oddball]

@kalbfled I see you mentioned it "might be" related and "maybe" isn't necessary anymore. Is there a simple way to validate to know for sure?

[kalbfled]

@mjones-oddball Sorry for the late response. I have the original code commented out in the lambda function. The simple way is to uncomment it, deploy the lambda, and see if it works.

[mjones-oddball]

@kalbfled based on your last comment it sounds like some changes are needed to the ticket description under the checkboxes section. Can you update please? Is there any information we need to provide around how to test this? If you took the ticket I understand you'd know how to do it, but another engineer might need more context.

[kalbfled]

updated

[cris-oddball]

@kalbfled we need a way to validate this. can you please add something about validation?

[kalbfled]

@kalbfled we need a way to validate this. can you please add something about validation?

Done.

[kalbfled]

I am duplicating the relevant code without the workaround in the new lambda for notification-kafka 3. If it works, we will have the answer for this ticket.

UPDATE: It didn't work.

[npmartin-oddball]

Keeping.