search

cityofaustin / atd-data-tech

Austin Transportation Data & Technology Services
17 stars 2 forks source link

Archive `atd-ctr-etl-production` out of EC2 to S3 and decommission it #18188

Closed frankhereford closed 1 month ago

frankhereford commented 1 month ago

With the recent closure of the of the data lake collaboration with the CTR @ UT, the EC2 machine atd-ctr-etl-production is no longer required. ~This issue is to track the archival of this machine via creation an AMI in EC2 and then archiving that AMI out to S3.~ Upon completion, then the machine should be terminated, being sure to clean up any EBS volumes, security groups, DNS entries and other similar orphaned resources.

Alternatively, if this issue is not accepted as a route forward, the services on the machine need to be migrated out of the CentOS7 environment currently on the host. An issue will need to be created in this case.

frankhereford commented 1 month ago

@mddilley asked if we have a S3 bucket in use for AMI archives, and we do not. This issue would include creating one.

mddilley commented 1 month ago

@frankhereford This morning, you mentioned that this server was terminated. If so, could you move this to In Progress and assign yourself?

frankhereford commented 1 month ago

❌ Route 53: transportation-data.austinmobility.io => CNAME => atd-ctr-alb-production-236583621.us-east-1.elb.amazonaws.com ❌ EC2 SG: atd-ctr-sg-ecs-ec2-production ❌ EC2 SG: atd-ctr-sg-ecs-ec2-etl-production

Unrelated entries which were obviously cruft: ❌ Route 53: atd-prefect-01.austinmobility.io => A => 3.90.149.203 ❌ Route 53: grafana.austinmobility.io => CNAME => atd-data03.austintexas.gov ❌ Route 53: ldm-test-graphql.austinmobility.io => A => 18.207.159.17 ❌ Route 53: ldm-staging-graphql.austinmobility.io => A => dualstack.ldm-staging-load-balancer-405449461.us-east-1.elb.amazonaws.com ❌ Route 53: ldm-test.austinmobility.io => A => 18.207.159.17 ❌ Route 53: rachel-workspace.austinmobility.io => A => 52.91.194.225 ❌ EC2 Keypair: rachel-ssh-key

frankhereford commented 1 month ago

I believe all the little things that we wanted to clean up here are gone, and I picked off some very obvious cruft while I was at it. Of note, I have /not/ deleted any of the data lake S3 buckets, and I recommend that we either spin those off into another issue to refine in an upcoming refinement, or make a larger "clean up AWS" issue which will include that and some other cruft we can discuss in refinement. @mddilley - how does that sound? Any way you want to go is good with me!