[Feature Request] - Run Amazon Linux 2023 as a virtual machine on premises

[Manouchehri]

Is your feature request related to a problem? Please describe. Creation of Amazon Linux 2022 virtual machine (VM) images for on-premises development and testing. e.g. the same as: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/amazon-linux-2-virtual-machine.html

Describe the solution you'd like This URL to be valid:

https://cdn.amazonlinux.com/os-images/2022/kvm/

Describe alternatives you've considered I can probably figure out how to make my own bootstrapped install using dnf install --installroot=/mnt, but it would be nice to have an official image to start with.

[ozbenh]

I'm bound to a VirtualBox environment, and have used the VB images from Amazon ( such as here https://cdn.amazonlinux.com/os-images/2.0.20231101.0/ ), so I guess I will wait while the bugs are ironed out of the kvm/vmware versions and see what happens. So glad to see movement on this again, though I did lose one client to the "dark side" on account of no offsite pre-deployment options like we had with the previous images. (I might try to pull them back if the VB image appears.) I'll take a quick peek once the URLs stop returning 404s. Thanks @stewartsmith.

The OVA can be imported into VB 6.1.48, but after change password there seems to be storage driver incompatibility; either the new password is not recognized upon reboot or error with "Authentication token manipulation". Will need to try 7.x and possibly tune the seed files...

VirtualBox has been .... weird ... can you try converting the image away from vmdk to vdi (I suppose there's a way to do that, I haven't researched) first ? I would not be surprised if VB has bugs around vmdk handling...

[tanvp112]

The VMDK formatted hard drive has no issue convert into VDI. Previously for AL2, a VB image is provided by AWS and it works out of the box. Believe it or not, I have run this copy & still running today for >5 years day-in-day-out. AL2 is arguably the best Linux experience I ever had so far.

The release of AL2023 image is a big step to many... kudos for AWS Linux team. Hopefully, there will be a VB image like the former and the superb experience would continue. For the case above, VB 6.1.x is sunsetting already, will need to test with latest VB 7.x instead.

Happy to hear more about others' experience using VB with the new image!

[yoonjs2]

Is possible to convert AL2023 arm64 qcow2 to parallels hdd format? I'm bounded to macOS with apple silicon environment and uses parallels in dev/production.

[ozbenh]

Is possible to convert AL2023 arm64 qcow2 to parallels hdd format? I'm bounded to macOS with apple silicon environment and uses parallels in dev/production.

I don't have a Mac to try (though @stewartsmith does, he might come up with an answer), but Google has a few hits on this. Alternatively there's a qemu for Mac which uses Hypervisor Framework under the hood and there's UTM as a front-end for it all

[ozbenh]

Is possible to convert AL2023 arm64 qcow2 to parallels hdd format? I'm bounded to macOS with apple silicon environment and uses parallels in dev/production.

I don't have a Mac to try (though @stewartsmith does, he might come up with an answer), but Google has a few hits on this. Alternatively there's a qemu for Mac which uses Hypervisor Framework under the hood and there's UTM as a front-end for it all

I did some experimentations and it didn't go particularly well. Not only converting to parallel is tricky and messy (the tools don't work well and parallels itself doesn't seem to be helping), but the end result doesn't boot.

I think I know why (they and vmware fusion both seem to be exposing storage as some kind of ACPI generated platform AHCI that isn't PCI backed, and we don't enable that driver). I'll see if I can get that turned on in a future kernel release. That said, we don't have plans at this stage to produce officially supported Parallels images.

[michailf]

Does anyone have a reference to login information? What user and password would I use for the images downloaded from the links provided?

[azN2]

Does anyone have a reference to login information? What user and password would I use for the images downloaded from the links provided?

Cloud-init

[stewartsmith]

Does anyone have a reference to login information? What user and password would I use for the images downloaded from the links provided?

Be sure to read the instructions on https://docs.aws.amazon.com/linux/al2023/ug/outside-ec2.html for configuring the VM. There isn't a password set, you need to set up something so that cloud-init can configure the instance on first boot.

[michailf]

I spent hours trying to figure out how to login to the provided image. I have read multiple documents, including the reference here. I cannot log in to the image. I was able to set the local hostname via the meta-data (guestinfo.metadata) so I believe I provide the data correctly. What I do not know is why the users are not being setup with a simple password.

Is there a working example of the user-data (guestinfo.userdata) that allows a simple password log in with sudo rights to the image provided?

[michailf]

I was able to log in using an ssh key. I then could create/adjust other users to my needs. It seems the cloud-init in the image does not allow for plain passwords.

[tanvp112]

I think I know why (they and vmware fusion both seem to be exposing storage as some kind of ACPI generated platform AHCI that isn't PCI backed, and we don't enable that driver).

This really rings a bell! I suspect this is required for the image to be compatible for VB as well.

[gebailey]

The VMDK formatted hard drive has no issue convert into VDI. Previously for AL2, a VB image is provided by AWS and it works out of the box.

I've built Vagrant cloud images (virtualbox provider) of Amazon Linux 2 for a few years now, and just uploaded one for Amazon Linux 2023 that was built from the VMware OVA. A few steps were required to convert from OVA->VMDK->VDI but seems to work just fine. https://app.vagrantup.com/gbailey/boxes/al2023

[ozbenh]

I was able to log in using an ssh key. I then could create/adjust other users to my needs. It seems the cloud-init in the image does not allow for plain passwords.

I have used images with passwords a few times but usually by creating a new user, something like:

  - name: benh
    groups: sudo
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
    ssh-authorized-keys:
      - <key>
    plain_text_passwd: <password>
    lock_passwd: false

This doesn't seem to work with ec2-user last I tried, but did you check if the old-school chpasswd method works ? From the AL2 on-prem docs:

chpasswd:
  list: |
    ec2-user:<plain_text_password>

[michailf]

I have used images with passwords a few times but usually by creating a new user, something like:

  - name: benh
    groups: sudo
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
    ssh-authorized-keys:
      - <key>
    plain_text_passwd: <password>
    lock_passwd: false

I got ssh-authorized-keys to work but not plain_text_passwd independently. I have not tried both of the options for the same user.

This doesn't seem to work with ec2-user last I tried, but did you check if the old-school chpasswd method works ? From the AL2 on-prem docs:

chpasswd:
  list: |
    ec2-user:<plain_text_password>

I also tried the chpasswd method but it did not work for me.

[ozbenh]

I just tested again with

#cloud-config
#vim:syntax=yaml
users:
  - name: test
    groups: sudo
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
    plain_text_passwd: passw0rd
    lock_passwd: false

and it worked fine

Are you logging in via the console or serial ? Or are you trying to ssh ? Passsword authentication is disabled in the ssh server by default.

[michailf]

I just tested again with

#cloud-config
#vim:syntax=yaml
users:
  - name: test
    groups: sudo
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
    plain_text_passwd: passw0rd
    lock_passwd: false

and it worked fine

I tried your example and it did work. I am not sure what I was doing wrong before.

The following example works to create a new user (al) with sudo access, enable the default user (ec2-user), set the ec2-user's password and force a password change on login.

#cloud-config
#vim:syntax=yaml
users:
  - default
  - name: al
    groups: sudo
    sudo: ALL=(ALL) NOPASSWD:ALL
    plain_text_passwd: Passwor8
    lock_passwd: false
chpasswd:
  list: |
    ec2-user:Passwor8

The following is a minimal example to gain full access to the system as ec2-user has sudo rights:

#cloud-config
#vim:syntax=yaml
users:
  - default
chpasswd:
  list: |
    ec2-user:Passwor8

[denisgabriel5]

Does anyone have some instructions on how to modify the VMware image according to this article?

[nileshtrivedi]

Is there a tutorial somewhere for running AL2023 on Apple M1 machines (perhaps via UTM)?

[ozbenh]

I did run for fun and no profit our qcow2 image on an M1 mac, I think using UTM a while back, though I don't remember the details. It's not a tested/supported solution so we don't have official docs, but I can try to snatch my partner's M1 one of these days and try again & let you know

[nileshtrivedi]

@ozbenh Actually I got it working with some effort. This is the user-data file that went into seed.iso attached as a CDROM in the UTM machine:

#cloud-config
#vim:syntax=yaml
password: test123
chpasswd:
  expire: False

users:
  - default
  - name: ec2-user
    lock_passwd: false
    plain_text_passwd: test123
    sudo: ALL=(ALL) NOPASSWD:ALL
    ssh_authorized_keys:
      - ssh-ed25519 yourkey_goes_here

It help to delete the .qcow2 and .iso disks fully and add again after every change. I think this is because most cloud-init config takes effect only on "first boot".

[yoonjs2]

For people looking for the al2023 vagrant box + apple silicon (arm64) + parallels desktop, I created and uploaded working box here: https://portal.cloud.hashicorp.com/services/vagrant/registries/hbsmith/boxes/al2023

I created the box using conversions on the mac like this: qemu qcow2 -> vmware vmdk -> parallels hdd then pack it into pvm & vagrant box. AL2023 has no problem with installing parallels tools extension and work with vagrant like a charm. (except manually assign fixed ip to network interface.)

Hope it helps.

[ozbenh]

Ah I was about to update here but you beat me to it :-) Yes, our ARM64 image seems to work with UTM and the default "virt" machine type (I think it's 7.2) on an M1. As for seed.iso , well ... that's documented in various places :-)

The one thing that I noticed doesn't work is Secure Boot as you can't enroll our keys, the problem seems to be that UTM comes up pre-provisioned with its own set of keys instead of in Setup Mode. It might be worth filing a feature request for that. We don't currently provide a pre-made UEFI variables blob in qemu format but you can use python3-virt-firmware or python-uefivars to convert ours from AWS format to Qemu format (you can find our blob and our keys in /usr/share/amazon-linux-sb-keys/, the UEFI variables blob is uefi.vars.

It would be nice if UTM provided AL2023 in their Gallery I suppose ...