DE Shutdown epic

[pdpinch]

We have been working on shutting down the DE network for some time, but there are still some lingering issues. This epic is for tracking them.

• transfer amps.ms.mit.edu subdomain from IS&T to Route 53
• [x] Set up redirect for http://amps-web.amps.ms.mit.edu/
• [x] redirect to Google form that send email to ZenDesk
• [x] Set up redirect (CNAME) for webcast.mit.edu to point to github pages
• [x] Set up https for amps-web.amps.ms.mit.edu
• [x] image techtv virtual hosts at is&t and shut them down ttv and ttv-dev
• [x] archive techtv AWS data (s3 and ec2 images)
• [x] send techtv AWS data to glacier
• [x] shutdown techtv AWS account
• Move Quantum tape drive to new location
• find storage for LTO back up tapes
• [x] move ttv GitHub repos
• [x] close ttv GitHub account

[shaidar]

Took a snapshot of the 2 instances running in ttvmit EC2 account and turned them off for now.

[blarghmatey]

The webcast.mit.edu domain is now directed at the github pages site and I filed a request for a zone transfer of amps.ms.mit.edu to a managed zone in route 53.

[shaidar]

@blarghmatey S3 objects in ttvmit account have already been moved to glacier storage class and to sync them to the current AWS account will require restoring them first to a different storage class.

Looks like the process needs to be: Glacier - Regular Storage - Sync to current AWS account - Move to glacier

[shaidar]

Opened AWS Support ticket to verify whether there's a better way of handling the S3 object sync than the conclusion I had reached a few days ago.

[shaidar]

AWS confirmed the conclusion we have reached as far as thawing anything in Glacier, syncing them over and then moving them into Glacier. I checked with Elaine and those are the buckets that we will be syncing:

• techtv-archive-staging
• techtv-restore-holding
• techtv-restore-logs
• ttv_bulkup
• ttv_error_logs
• ttv_logs
• ttv_static
• ttv_uploads
• ttv_videos

[shaidar]

Still syncing buckets since Friday. ttv_videos has synced 9.3TB so far.

[pdpinch]

Hm. How much do we have to go? It doesn't sound like this will be done before the end of the month...

[shaidar]

@pdpinch It's 60735 Objects with a total size of 27.7 TB

[shaidar]

We're at 16TB now. Also after talking with Elaine, we will not be thawing and syncing techtv-archive-staging as it contains a subset of the files in already in ttv_videos.

[shaidar]

@pdpinch We're at 18.4TB and don't think it will be complete by Friday.

[pdpinch]

Ok. Elaine spoke with finance and they said it's OK to go over a couple of days.

We still want to shut down the AWS account asap.

@pdpinch https://github.com/pdpinch We're at 18.4TB and don't think it will be complete by Friday.

[shaidar]

ttv_videos finally completed syncing. Doing a final check on AWS resources to make sure we didn't miss anything and then we can close out the account.

[shaidar]

AWS TechTV account closed on 07/03/18

[shaidar]

Discovered that amps subdomain was in fact delegated to two amps name servers. After verifying with Elaine that nothing is needed under that subdomain except for the amps-web name and that the name servers were part of what was going to be retired, we asked IS&T to remove the NS delegations for the zone and added a single CNAME for amps-web.amps.ms.mit.edu to point to amps.odl.mit.edu. amps.odl in turn has a CNAME pointing to an EC2 running nginx and redirecting all requests to a Google form that Elaine set up.

[pdpinch]

@emmello decided to leave the Quantum tape system where it is for now. There isn't any immediate pressure to clear out of that space yet.

[pdpinch]

The TechTV org and repos on GitHub have been deleted. All the repos have been moved to GHE.

[pdpinch]

I added a checkbox for setting up https on amps-web.amps.ms.mit.edu. This is blocked by the letsencrypt salt formula, I think?

[shaidar]

Deployed letsencrypt manually as we work through the salt stack formula for it. Also set it up to auto-renew, so we should be good.

[blarghmatey]

@pdpinch looks like we can close this now...

[pdpinch]

👍