openml / automlbenchmark

OpenML AutoML Benchmarking Framework
https://openml.github.io/automlbenchmark
MIT License
391 stars 130 forks source link

EC2 AMIs: Outdated #512

Closed Innixma closed 1 year ago

Innixma commented 1 year ago

The current EC2 AMis used in AMLB are very old (I believe they were created in ~2019):

  ec2:
    regions:                             #
      us-east-1:
        ami: ami-0ac019f4fcb7cb7e6
        description: Ubuntu Server 18.04 LTS (HVM), EBS General Purpose (SSD) VolumeType
      us-west-1:
        ami: ami-063aa838bd7631e0b
        description: Ubuntu Server 18.04 LTS (HVM), EBS General Purpose (SSD) VolumeType
      eu-west-1:
        ami: ami-00035f41c82244dab
        description: Ubuntu Server 18.04 LTS (HVM), EBS General Purpose (SSD) VolumeType
      eu-central-1:
        ami: ami-0bdf93799014acdc4
        description: Ubuntu Server 18.04 LTS (HVM), EBS General Purpose (SSD) VolumeType

This leads to them having security vulnerabilities and thus are not able to be SSH'ed into when using an AWS account with proper security controls.

I would strongly recommend these AMI's be updated to 2023 AMIs.

I have tested the following us-east-1 AMI which runs successfully on AutoGluon:

    # Note: Added a new updated region image, old images no longer responded to SSH and didn't work correctly
    regions:                             #
      us-east-1:
        ami: ami-09cd747c78a9add63
        description: Canonical, Ubuntu, 20.04 LTS, amd64 focal image build on 2023-02-07

I found this updated AMI by:

  1. Going to https://us-east-1.console.aws.amazon.com/ec2/home?region=us-east-1#LaunchInstances:
  2. Clicking 'Browse more AMIs'
  3. Under 'Quickstart AMIs', selecting 'Ubuntu Server 20.04 LTS (HVM), SSD Volume Type'
  4. In the AMI from catalog view, you can see the AMI ID:
    
    Amazon Machine Image (AMI)

ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20230207 ami-09cd747c78a9add63

-> 

ami-09cd747c78a9add63


We can see that this is a recently created AMI:

Published

2023-02-07T22:43:00.000Z


We should be able to repeat this process for every region we want to support, and then update the `config.yaml` with the new AMIs.

Note: I updated to 20.04 from 18.04, but there is also an updated 18.04 AMI image too in case using Ubuntu 18.04 is preferred over 20.04.

ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-20230131 ami-0263e4deb427da90e

Published

2023-01-31T16:36:13.000Z

PGijsbers commented 1 year ago

Is there a good reason to go with 20.04 over 22.04? The latter has two additional years of support which sounds great. So we would instead use ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-20230208(ami-0557a15b87f6559cf) or equivalent.

Innixma commented 1 year ago

I think 22.04 should be fine, I don't personally know the difference between the versions of Ubuntu.