[ci.jenkins.io] Move controller (VM) to AWS

[dduportal]

This issue tracks the work to migrate ci.jenkins.io controller to the AWS Sponsored.

Current resources:

• User facing documentation: https://github.com/jenkins-infra/documentation/blob/main/ci.adoc
• Terraform definition of the current VM: https://github.com/jenkins-infra/azure/blob/main/ci.jenkins.io.tf
  • Utilizes a Terraform module: https://github.com/jenkins-infra/shared-tools/tree/main/terraform/modules/azure-jenkinsinfra-controller
  • With a dedicated subnet: https://github.com/jenkins-infra/azure-net/blob/5e16478469344f8eced5224e553a5f660794b9dd/vnets.tf#L204-L209 (there is a former subnet for the controller in the public_vnet for it but it is not used anymore)
  • 2 IPs (public for HTTP/HTTPS/JNLP and private for SSH) - https://github.com/jenkins-infra/azure/blob/e81b9697e4a9cedebf10e119b6a5a112e09b651f/ci.jenkins.io.tf#L57-L60
• VM is provisioned with puppet: https://github.com/jenkins-infra/jenkins-infra/blob/bb9e757b20dcec1da678948ef8b453d6d2cac928/manifests/site.pp#L77-L85

Target for the new VM:

• Same size (vCPUs and memory)
• Same disk topology (rootfs and data with jenkins home in it)
• For network: see https://github.com/jenkins-infra/helpdesk/issues/4320
  • We'll need a aws.ci.jenkins.io DNS A record (pointed by the CNAME ci.jenkins.io once migrated) to the public IPv4 of the controller, and a AAAA record with the public IPv6.
  • If need be, we'll define a private DNS A record aws.ci.jenkins.io so agents can reach the controller through private subnets
  • Inbound:
  • Anywhere HTTP + HTTPS on both IPv4 and IPv6
  • SSH from the (Azure private VPN) only. We'll set up the VPN routing of users to override routing like we did for the pkg.origin.jenkins.io, usage.jio and census.jio VMs.
  • Private subnets HTTP, HTTPS, JNLP
  • Outbound (IPv4 only):
  • HTTP+ HTTPS + HKP to everywhere
  • SSH to GitHub public IPs and to private subnets

[dduportal]

Discussed with @smerle33:

• https://github.com/jenkins-infra/helpdesk/issues/4320 reached a first milestone allowing starting work on the issue here
• Stephane prefers more the EKS topic than the VM topic: assigning VM to @dduportal
• Scheduling for current milestone

[dduportal]

Update:

• We need a DNS zone in AWS, delegated from our Azure DNS, to ensure we keep the DNS records up to date when changing IPs

• We want full in and out IPv6 support

• Controller must be in a public subnet, with both Network ACL and security groups restrictions.

• Big PR network + DNS + reports in https://github.com/jenkins-infra/terraform-aws-sponsorship/pull/36

• Initial DNS delegation to validate it works in https://github.com/jenkins-infra/azure-net/pull/317 (need an update with the new DNS zone name servers => automation incoming)

• Initial LDAP allow list update (need an update with the new DNS zone name servers => automation incoming) in https://github.com/jenkins-infra/kubernetes-management/pull/5891