Move private key storage (for login) to credentials service and make plan for rotating private keys

[mjtravers]

As part of the Login.gov authentication process, private key certificates need to be stored server-side in a secure but easily accessible place. Environment variables in Cloud.gov have proven to be impractical for this purpose. A possible solution is to store the key files in the credentials service or into AWS S3.

As part of this effort:

• [x] Move the private key certificates into the credentials service (S3 if this is not possible)

• [ ] Develop a strategy for monitoring expiration dates of private keys for login / rotating private keys without downtime (current keys expire June 2024)

  | FECFile Web API (DEV) | 2023-06-27 |   | FECFile Web API (STAGE) | 2023-08-10 |   | FECFile Web API (PROD) | 2024-11-28 |

• [x] Verify no other sensitive info is outside the credentials service

Environment variable with SSL key is: OIDC_RP_CLIENT_SECRET

Technical Notes

First option is credentials service. Secondary option is S3.

If S3 option

• Consider asking cloud.gov about security Cloud.gov S3 info
• AWS storing private key info
• Look into a management process to automate the creation of the key files and pushing them to S3
• Login.gov will handle more than one so look into having a backup key

Acceptance Criteria

• Users can log in
• App is retrieving the private key from crediential service or S3 but not the ENV vars

QA Notes

null

DEV Notes

null

Design

null

FECFILE-331

[mjtravers]

We have the Cloud.gov credentials service has been set up, see ticket #56. The credential service already has the OIDC_RP_CLIENT_ID variable in it so the question is can we move the OIDC_RP_CLIENT_SECRET environment variable into the credential service as well. When this ticket is picked up, let's ask Laura if there was a reason the OIDC_RP_CLIENT_SECRET was not moved into the credentials service and if we could move it there rather than into an S3 service.

[mjtravers]

Comment from @lbeaufort in #703 which was merged into this ticket:

From https://developers.login.gov/production/#certificate-rotation-process:

Certificate Rotation Process If you are rotating your application’s public/private keypair, or want to add an additional public certificate, please follow the instructions below to add the new certificate to your application’s Login.gov configuration.

Please note that from the time you notify Login.gov of your intent to rotate certificates, it can take up to 2 weeks for the new certificate to be available in production.

Follow this process to initiate certificate rotation: For OIDC integrations or SAML integrations sending signed requests:

• Add the new certificate to the application dashboard configuration.
• Contact Login.gov technical support and request certificate addition.
• Once certificate deployment is confirmed, rotate the key pair at your convenience.
• Once the new key pair is in use, please submit a request to remove the old certificate.

Also, found this helpful documentation for another project: https://github.com/GSA/data.gov/wiki/Login.gov-SAML-certificate-rotation-steps

[dheitzer]

Confirmed with the team during discussion that we would like this ticket to automate the key rotation process, ideally with the ability to kick it off from an fecfile-web-api django-admin management command. This command would need to do the following

1) generate the public/private key pair. 2) upload the public key to login.gov (possibly via api call to an endpoint retrieved from their discovery service, if it exists). NOTE: we will need to confirm if login.gov will deploy the certificate at this point or if there will be a wait time before the next steps can be taken. 3) prepare private key for upload to the credential service. 4) update the credential service service with the new private key. 5) delete the public/private key from the file system and have command cleanup after itself. 6) restart the api to allow new settings to take effect

[dheitzer]

After discussing with @lbeaufort, in order to minimize the manual steps required to rotate the keys the following implementation will be made:

1. create a django-admin management 'key generation' command to generation of public/private key pair to use for our login.gov app.

   • The credential service will be used to store potentially sensitive -subj values/attributes for use in the key generation as documented in the project login.gov setup notes and will be used upon key generation.
   • Openssl or whatever tool that is used for the generation of the key pairs will need to be added to the requirements.txt file to allow tracking via Snyk.
   • The generated public key will need to be programatically stored in a secure S3 bucket to allow the administrators to upload to login.gov without storing it at rest in the container filesystem.
   • The generated private key will need to be transformed into appropriate text and staged in the credential service in preparation for activation (in a JSON attribute that is separate from the live private key JSON attribute).
   • The command should instruct the user to upload the key to login.gov and to notify them accordingly, and to execute the subsequent install django-admin command when it is ready.

2. create a django-admin management 'key installation' command to install the staged private key (from step 1) into the api and restart it.

   • This command should move the existing private key in the credential service (but not delete it) and move the staged key to the live JSON attribute.
   • This command should delete the public key in the secure S3 bucket once it is no longer needed.
   • This command should restart the api to allow it to take effect.

3. create a django-admin management 'key backout' command to backout the current private key and replace it with the original.

   • This command should move the backup key JSON attribute to the live JSON attribute to revert any key update.
   • This command should restart the api to allow it to take effect.