Concourse for Pages

[bengerman13]

In order to have compliant ci, the Pages team needs to use concourse.

Acceptance Criteria

• [x] Credential storage: We currently store credentials in S3 and inject them during set-pipeline calls. This is not ideal for the Pages team as they would need access to S3 buckets. The cg ops team has also discussed moving away from using S3 for ease of use and to remove sensitive information from our pipelines. Concourse + Credhub integration is desired as Concourse will fetch credentials at runtime, keeping creds completely out of pipelines.

• [x] Authentication: Currently, cg operators authenticate via opslogin.fr.cloud.gov and are granted access via the existence of the concourse.admin scope. The Pages team can be added to OpsLogin and individually assigned to roles in Concourse.

• [x] Authorization: Current cg operators are all administrators of Concourse. The Pages team only needs access to their pipelines and their credentials following the principles of least privilege.

  ---

Security considerations

See https://github.com/cloud-gov/internal-docs/blob/master/design-decisions/concourse-pages.md. This outlines the required authentication and authorization controls.

Implementation sketch

See: https://github.com/cloud-gov/internal-docs/blob/main/concourse/concourse-pages.md

Staging

• [x] Add a local admin user to prevent lockout (https://github.com/cloud-gov/internal-docs/blob/master/runbooks/concourse/default-local-account.md)
• [x] Pipeline to deploy Credhub (https://github.com/cloud-gov/deploy-credhub)
  • [x] DNS: credhub.fr(-stage).cloud.gov
  • [x] ALB rule
  • [x] Use OpsUAA for authentication
  • [x] Test ACLs
  • [x] Pages team ACLs (use terraform)
  • [x] Hook up new Credhub to doomsday
• [x] Configure concourse to talk to credhub
  • [x] Verify no existing pipelines are affected
• [x] Configure a Pages team in Concourse
  • [x] Establish RBAC using OpsLogin
  • [x] Test team pipeline
• [ ] Remove credhub subnet from cg-provision (deploying to the concourse subnet)

To be continued...

Production

• [ ] Set the local account creds in credhub and s3
• [ ] Add local account by uncommenting ops file in pipeline.yml

[davemcorwin]

FYI - I am not able to edit the issue.

1. Secrets Management - yes
2. VPN Only - Currently, nothing we do requires the VPN so I'm not sure how to answer this. If this is a requirement for FISMA moderate then we gotta do what we gotta do.
3. Would the setup be Terraform-able? This probably relates more to wherever our credentials are stored, I do see a Concourse provider for Terraform and I am currently using the Cloud Foundry one. The use case here is for injecting environment variables from resources in AWS created via Terraform. For example, we create an ECR instance in AWS via Terraform and need the credentials for associated IAM users when deploying the image to cloud.gov. This CAN be done manually, but is much much nicer when automated.

[bengerman13]

for 3 - I'm not sure I understand the question, but I think it is "will concourse pipelines/tasks be able to leverage data from terraform" If that's right, we do this a couple places and are happy to show you the patterns we use there.

As far as edit-ability on this ticket, I mentioned this during standup. I think we need to get the federalist team access to some/all of our github repos. Kara mentioned an upcoming meeting with the federalist team, her, and Peter that would be good to discuss those permissions at

[davemcorwin]

for 3 - I'm not sure I understand the question, but I think it is "will concourse pipelines/tasks be able to leverage data from terraform" If that's right, we do this a couple places and are happy to show you the patterns we use there.

Exactly, thanks!!

As far as edit-ability on this ticket, I mentioned this during standup. I think we need to get the federalist team access to some/all of our github repos. Kara mentioned an upcoming meeting with the federalist team, her, and Peter that would be good to discuss those permissions at

Sounds good, thanks!!!

[spgreenberg]

You will likely also need an S3 bucket in which to store artifacts and version files.

[pburkholder]

We're predicating this on the assumption that a 3rd concourse is not "too much" work. If this is going to get way over say, 16 or 24 hours, then wave a flag and we'll revisit.

[spgreenberg]

Current status:

• Pages team is in staging concourse.
• Pages clients are created in staging cf (and working w/ the pages apps).
• Docs are updated: https://github.com/cloud-gov/internal-docs/blob/main/concourse/concourse-pages.md & https://github.com/cloud-gov/internal-docs/blob/main/concourse/teams.md
• Credhub is deployed to staging but not yet integrated and operational.

[bengerman13]

for the credhub step above Use OpsUAA for ACLs - credhub doesn't currently support anything but user name/id for ACLs, so we need to manage this on a user-by-user basis, either by populating usernames in the bosh deployment, or by adding users with the credhub CLI.

[bengerman13]

I think the next step is to follow the instructions here to hook concourse up to credhub then follow the instruction on the same page to hook credhub up to opsuaa then write the Terraform config to manage credhub ACLs

[bengerman13]

Added credhub.read and credhub.write scope and authority to the concourse client in opsuaa. Opened PR to add credhub.yml opsfile to concourse-staging. Next step (once that PR is merged) is to link variables in tooling credhub. variable names here

this will finish hooking credhub, opsuaa, and concourse-staging up together, then we can move on to ACLs

[bengerman13]

PR for credhub + concourse updated to link up all the variables.

I'm working on figuring out how to hook terraform up in this pipeline - we need to use a TF provider that's not in the marketplace, so we need to pass it to the terraform concourse resource before calling terraform. Once that's sorted out, we can add the actual TF to do this, and ship it.

[soutenniza]

https://github.com/orange-cloudfoundry/terraform-provider-credhub was installed via https://github.com/cloud-gov/cg-deploy-concourse-docker-image/blob/main/task/concourse-tasks/scripts/build.sh#L133-L135

Note: version 0.13.3 was only that worked for installation.

The staging pipeline has terraform apply sorta working with this: https://github.com/cloud-gov/deploy-credhub/blob/initial-deploy/terraform/acl.tf

The issue currently is trying to make Terraform 14 actually finding the provider that was installed.

I have tried this: https://www.hashicorp.com/blog/automatic-installation-of-third-party-providers-with-terraform-0-13 with no luck.

I did try adding a filesystem_mirror for the terraformrc similar to this: https://www.terraform.io/docs/language/providers/requirements.html#in-house-providers but it couldn't find the provider for some reason.

I will pick this up after vacation but my notes above if anyone wants to pick it up