search

lttkgp / C-3PO

The metadata overlord and API server for LTTKGP
https://api.lttkgp.com
MIT License
27 stars 22 forks source link

Write a lambda function to rotate Spotify Access Token #89

Closed mukul-mehta closed 3 years ago

mukul-mehta commented 3 years ago

For now, we've setup the SPOTIPY_CLIENT_ID and SPOTIPY_CLIENT_SECRET environment variables in our AWS Secrets Manager. Since these tokens expire and as a good practice, it'd be nice to setup automatic rotation of these every fixed number of days. AWS Secrets Manager allows for writing a Lambda function to rotate keys and provide the new keys back to it which it updates automatically.

For Spotify,here's their guide to authorization workflow: https://developer.spotify.com/documentation/general/guides/authorization-guide/#client-credentials-flow

andrewmiko commented 3 years ago

@mukul-mehta Hey, I can work on this one. I'm not sure, but I think that this should be separate Python project (repo) with own .env, settings, etc. I can use https://github.com/aws/chalice to create this one. What do you think?

mukul-mehta commented 3 years ago

Hi @andrewmiko Thank You for your interest!

The library seems great, you could use it to write a lambda function to rotate keys. I'm not sure if we should create a new repository for this task though, I believe we could write them all in this repository. I'm assuming, the only environment variables we'll need are AWS keys which we already manage using GitHub. I believe it might be a good idea to start with writing the functions in this repo and then we could see if we want to create a separate repo with a collection of AWS Lambda functions used org-wide

andrewmiko commented 3 years ago

@mukul-mehta Good.

Please assign this one on me. I will take care of that.

andrewmiko commented 3 years ago

@mukul-mehta after small review of the project I've several questions about integrating Lambda function.

  1. Folder structure According to https://aws.github.io/chalice/topics/packaging.html guide of Chalice app packaging, I assume that we need to create separate folder where will be: app.py and separate requirements.txt for Lambda functions. Do you agree or I have missed something here?

  2. Environment variables Environment variables are loaded into the project during project start from here: https://github.com/lttkgp/C-3PO/blob/a93369c49ceaf66c2f2832ee8656b9d982024a50/wsgi.py#L6 So then to load those variables into Chalice requirements.txt I will load them directly into app.py using load_dotenv function. What do you think?

  3. Deployment Deployment of Chalice function should be done separately from new folder where Chalice files are. Later It can be deployed with separate pipeline. Any concerns about this one?

  4. AWS keys Chalice will require AWS keys according to this guide: https://aws.github.io/chalice/quickstart.html#credentials. Do you have those keys which I can use? Do you have those keys somewhere in CI/CD which can be automatically used?

Thanks.

ghostwriternr commented 3 years ago

Hey @andrewmiko! Thank you so much for sharing detailed explanations to all your questions. Much much appreciated!

I believe a separate repo would indeed be a good idea since it would let the function not be tied to this project and be used by absolutely any service. This would also answer your questions 2-3.

This being said though, AWS actually provides templates for all sorts of keys on Secrets Manager, which we can use. So we won't really be needing a full-blown microframework like Chalice to do the task. The templates are particularly helpful because they correspond to the excellent documentation AWS has around the lifecycle of a key-rotation function. Relevant template for our use case: lambda_function.py

A good call-out on #4 as well. We can create an IAM user on our account with the required permissions and use it for the lambda function deployment.

ghostwriternr commented 3 years ago

Also, would you mind picking up a completely identical issue here: lttkgp/R2-D2#14. It is only during the course of this discussion that we realized we don't yet have the need to rotate Spotify API keys (since we're only accessing public resources) as they never expire. But we use Facebook's Graph API keys heavily and they do expire. I have created a new repo here you can use in case you wish to pick this up: https://github.com/lttkgp/graphapi-key-rotation

I'm keeping this issue here around till you respond : ) Apologies for the confusion!

andrewmiko commented 3 years ago

@ghostwriternr thanks for info. Do you need this issue? Maybe close it? I can move forward and look into lttkgp/R2-D2#14 this one. Thanks.