Document how to prepare EC2 for running remote dev containers

[tschaffter]

In AWS Management Console

• Instantiate EC2 instance
  • Name and tags
  • Name: <GitHub username>-devcontainers
  • Tags:
    • Department: IBC or CNB (selected from this list)
    • Project: challenge (selected from this list)
    • OwnerEmail: <your email address>
    • CostCenter: NIH-ITCR / 101600 (selected from these lists)
  • Application and OS Images (Amazon Machine Image)
  • Name: Ubuntu Server 22.04 LTS (HVM), SSD Volume Type
  • Architecture: 64-bit (x86)
  • AMI ID: ami-09d56f8956ab235b3
  • Instance type
  • Instance type: t3a.xlarge
  • Network settings
  • Click on Edit
  • VPC: vpc-0e9b80dc470a797d5 (sandcastlevpc)
  • Subnet: subnet-025c297e427e44daf (Sandcastle-Private1)
  • Firewall
    • Click on Select existing security group
    • Select security group challenge-registry-devcontainers (sg-03b891e56d6a1e851)
  • Configure storage
  • 1x 50 GB (gp2)
• Identify the Private IPv4 addresses of the EC2 once it has started

On your local host

• Add an host profile to your local .ssh/config

  Host devcontainers
     HostName <private_ip>
     User ubuntu
     IdentityFile ~/.ssh/tschaffter-sandbox.pem

• Connect to Sage VPN
• SSH to the instance

  ssh devcontainers

On the EC2 instance

• Update system packages

  sudo apt update && sudo apt upgrade -y

• Install Docker Engine on Ubuntu
• Add your user account to docker group.

  sudo usermod -aG docker ${USER}

• To apply the new group membership, log out of the server and back in. Run the command groups to check that your user is a member of the group docker.
• Verify that Docker Engine is installed correctly by running the hello-world image.

  docker run --rm hello-world

• Clone your fork in the home directory

In VS Code

• Install the extension Remote - SSH and Remote - Containers
• Remote-SSH: Connect to Host... > Select the host
• Verify that the green button on the bottom-left corner shows SSH: <host name> upon successfully connecting to the remote instance.
• File > Open Folder... > Select the folder of the project
• Remote-Containers: Open Folder in Container...
• Click on OK to open the project folder in the dev container
• Verify that the green button on the bottom-left corner shows Dev Container: Challenge Registry @ ssh://<host name>

Congratulations, you are now ready to develop in the dev container that runs on the EC2 instance! 🚀

[tschaffter]

I can now ping the devcontainer after adding this inbound rule:

Type: Custom ICMP rule
Protocol: Echo Request
Port: N/A
Source: your choice (I would select Anywhere to be able to ping from any machine)

[tschaffter]

test.challenge-registry.org and my devcontainer EC2 have virtually the same security group. One may difference is that the dockerized app on the test server is running on the host's docker engine, while I'm currently running the containers in the devcontainer using the VS Code option docker-in-docker.

I tried the option docker-from-docker that now makes the devcontainer use the host's docker engine. I can now successfully start the app and access it my copy-pasting the private IP address in my browser.

A simple solution is to use docker-from-docker instead of docker-in-docker. Using an EC2 provide in itself all the isolation we need from our local computer.

Here are the containers running on the EC2:

$ docker ps
CONTAINER ID   IMAGE                                                                  COMMAND                  CREATED          STATUS          PORTS                                           NAMES
c99141320412   sagebionetworks/challenge-registry-web-app:latest                      "/docker-entrypoint.…"   12 seconds ago   Up 10 seconds   0.0.0.0:80->80/tcp, :::80->80/tcp               challenge-registry-web-app
06492c258030   sagebionetworks/challenge-registry-api:latest                          "/docker-entrypoint.…"   12 seconds ago   Up 11 seconds   0.0.0.0:8080->8080/tcp, :::8080->8080/tcp       challenge-registry-api
8a61ce97eb44   sagebionetworks/challenge-registry-api-db:latest                       "docker-entrypoint.s…"   12 seconds ago   Up 11 seconds   0.0.0.0:27017->27017/tcp, :::27017->27017/tcp   challenge-registry-api-db
3cd9a49270cc   vsc-challenge-registry-0fa1610b06cbe44dd8da6b5412593805-features-uid   "/bin/sh -c 'echo Co…"   7 minutes ago    Up 7 minutes                                                    quizzical_gates

[tschaffter]

Exploring how VS Code forward port with the option docker-in-docker. The following port info is visible after starting the devcontainer. None of the stack containers are running yet.

Now I start the stack with docker compose up -d:

[tschaffter]

Note about the port issue

I realized that I can access the challenge registry running INSIDE the devcontainer on the EC2 using localhost (using docker-in-docker). 🤯