turnerlabs / airflow_stacks

Some airflow stacks we are working on at Turner
GNU General Public License v3.0
3 stars 0 forks source link

Improve secrets manager integration #83

Closed smithatlanta closed 5 years ago

smithatlanta commented 5 years ago

Currently every time the website is hit or the scheduler runs, the connect.sh script gets called to get the RDS connection. This is making many AWS api call to secret manager which is slowing things down. I would like to change this to where the connect.sh grabs the connection from an environment variable that is set by a cron job that makes the AWS api call every 5 minutes.