Bug: API lambda SSM parameter init is brittle

[patheard]

Summary

The API lambda function relies on its entry.sh script to load secret values from SSM ParameterStore: https://github.com/cds-snc/url-shortener/blob/7f370f15b930e710a37680faef99c069ec664567/api/bin/entry.sh#L45-L100

The problem this creates is that if the entry.sh fails to load any parameters but starts the lambda function, that function is now unable to serve requests and throws errors.

The SSM ParameterStore init should be moved into the Python code to surface errors cleanly and allow for function restarts.

[patheard]

Looking into it, using Pydantic settings management will work nicely for this: https://docs.pydantic.dev/latest/usage/settings/

We should be able to do the following:

1. Centralize all environment config retrieval in a single Settings class that gets initialized outside the Lambda handler.
2. Add a customized settings source for SSM ParameterStore and give precedence to the env var source.

For the custom settings source we could either use an existing module like pydantic-ssm-settings or write our own integration (SecretsManager example).

[patheard]

With the changes to the entry.sh this is now self-healing if the secret retrieval fails on init. Future invocations of the function will keep trying to load the secrets until they succeed.

Probably still a good idea to look at using something like Pydantic to get away from the bash though.

[patheard]

Closing for now as brittle inits have been fixed.