CDCgov / data-exchange-hl7

Enterprise Data Exchange (DEX) is a new cloud-native centralized data ingestion, validation, and observation service scoped for common data types (HL7, FHIR, CDA, XML, CSV) sent to the CDC. It helps public health stakeholders who send data to the CDC while reducing the maintenance efforts, complexity, and duplication of ingestion points to CDC.
Apache License 2.0
10 stars 14 forks source link

Need self-hosted Github runner for Terraform #143

Closed boris-ning-usds closed 1 year ago

boris-ning-usds commented 1 year ago

We're starting to run into issues when running Terraform outside of the network (using public Github actions) that we're provisioning infrastructure for.

For example, since our Azure Vault (secret storage) is blocking all public access, Github actions (from public internet) won't be able to pull Azure Vault secrets to put as Azure function parameter for our functions to properly deploy.

I would like to look into running self-hosted Github runners in Azure so that Terraform can run as part of that. I've started initial conversations on how to do this.

Method 1

Method 2

boris-ning-usds commented 1 year ago

I created a subnet of size /28 (11 available IPs, 5 used up by Azure) for the creation of this virtual machine in the development environment.

I filed REQ0055450 as a Cloud Service Request (https://servicedesk.cdc.gov) to get the virtual machine provisioned and stood up.

boris-ning-usds commented 1 year ago

Ticket update: still waiting for Cloud team to create that virtual machine.

rmharrison commented 1 year ago

Status, end of Sprint 22 Sep - 5 Oct: Waiting for Cloud team

boris-ning-usds commented 1 year ago

:( I heard I needed another team to help, and I needed to file a ticket via a different portal with a virtual machine form fill out.... for one virtual machine.

Sadness and re-learning what the process is to get a VM spun up for running a terraform apply job. This ticket will need to be move to next sprint.

boris-ning-usds commented 1 year ago

Creating C73589 to supplement existing Cloud Service Request ticket to get us a self-hosted Github runner VM.

aktech commented 1 year ago

By the way: cirun.io does the same, without adding maintenance burden.

boris-ning-usds commented 1 year ago

From the US federal government standpoint, the service above is not registered as part of FedRAMP marketplace and would require a lot more compliance to go through. It could be true that it's less maintenance burden than having to spin up my own VM - but it would incur a lot more compliance burden.

aktech commented 1 year ago

Ah, I see. Thanks for letting me know. I'll take a look at it anyway.

boris-ning-usds commented 1 year ago

I just got the virtual machine provisioned - looking into the boundaries of it and setting it up for Github integrations now.

boris-ning-usds commented 1 year ago

This is now completed. I'm running terraform code via a service setup with a simple VM in our environment. Started this ticket: #193 for any deficiencies incurred setting this VM up.

Will be demo-ing this to the team this sprint.