Private EC2

[matusdrobuliak66]

Currently, our EC2 instances are publicly available.

1. Create Private Subnets: Move our Docker Swarm nodes to private subnets. Private subnets don't have a route to the internet by default and their instances can't be accessed directly from the internet.
2. Set Up NAT Gateway: Create a NAT Gateway in a public subnet. This NAT Gateway allows instances in the private subnet to initiate outbound connections to the internet while hiding their private IP addresses.
3. Update Route Tables: Update the route tables of your private subnets to route outbound traffic to the NAT Gateway.
4. Security Group Configuration: Ensure that your security groups only allow necessary inbound and outbound traffic. For example, you might allow inbound traffic only from specific IP addresses or subnets that need to access your services.

By setting up the Docker Swarm nodes in private subnets and routing their outbound traffic through a NAT Gateway, we effectively hide them from direct access from the internet. This setup adds an additional layer of security to our infrastructure.

### Done in stages
- [x] Master --> https://git.speag.com/oSparc/osparc-ops-deployment-configuration/-/merge_requests/664
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/issues/782
- [x] [Prod] AWS TIP --> https://git.speag.com/oSparc/osparc-ops-deployment-configuration/-/merge_requests/674#e7d14d429f898757a423156de0e8c49d75240694
- [ ] [Prod] NIH
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/issues/822

[mrnicegyu11]

connected to https://github.com/ITISFoundation/osparc-ops-environments/issues/195

[YuryHrytsuk]

Conclusions

We can have Cluster EC2 Instances running in private subnets and being reachable via AWS NLB (source: SO Answer)

[YuryHrytsuk]

Steps

• Configure ansible to work with Bastion host
• Prepare terraform
• Backup data (+ SSL Certs)
• Run terraform
• Deploy services
• Apply backups (+ SSL Certs)

Issues encountered

• [ ] configure_kubernetes step fails (fail to restart kubelet and awaits for privilage escalation). Works locally on a different ansible verison
• [ ] make helmfile-diff fails if nothing is deployed (could be related to CRDs installation)
  • The issue got fixed if you run helmfile sync manually (2x times). Then it got resolved.
• [x] if k8s plan fails, no simcore services can be deployed
• [X] bridge docker networks uses default pool and not the one we configure (old issue)

p.s. checked items are already fixed in code :)

Manual steps

• traefik certs needs to be copied manually
• after terraform apply we need to run ansible-playbook

Side-effects to fix

• [X] docker networks test is broken
• [x] test machines e2e-ops test is broken

[YuryHrytsuk]

Blocked by https://github.com/ITISFoundation/osparc-ops-environments/issues/716

[YuryHrytsuk]

Also blocked by https://github.com/ITISFoundation/osparc-ops-environments/issues/718