aws-samples / aws-health-aware

AHA is an incident management & communication framework to provide real-time alert customers when there are active AWS event(s). For customers with AWS Organizations, customers can get aggregated active account level events of all the accounts in the Organization. Customers not using AWS Organizations still benefit alerting at the account level.
MIT No Attribution
348 stars 143 forks source link

Lambda cold/warm start issue when DNS changes region #68

Open geralexgr opened 1 year ago

geralexgr commented 1 year ago

There is an issue in the code when the DNS is switching from one region to another.

We have noticed that when there is a failover from one region to another from the loadbalancer, although the health_active_region is changing, due to the fact that codeblock is not wrapped into a function or in the function handler, the code is not in warm state and wrong results are returned. Currently the initialisation of the config is done on the cold state. As a result when one runs the lambda every x minutes when a failover occurs it will return an error.

In the function logs we notice an error message like:

Client is configured with the deprecated endpoint: us-east-2

https://github.com/aws-samples/aws-health-aware/blob/6f047af3ec397e1cb5896f3c96dde9893d4bf880/handler.py#L24-L38

A solution for this issue could be to wrap the initialisations inside a function.