`amazon-ec2-amis.nix` is out of date and 23.11 AMIs have not been built yet

[samuela]

Issue description

This issue is two-fold:

1. 23.11 AMIs have not been generated yet, despite 23.11 having been released a few months ago.
2. The list of AMIs we maintain in code does not contain all AMIs that do exist on AWS.

If this blog post is to be believed, NixOS publishes official AMIs under the owner owner-id=080433136561. (Side issue: it would be helpful if the NixOS foundation could formally testify to this somewhere publicly, so that users don't have to rely on blog post hearsay.)

Pilfering some jq-fu from that blog post, we can find AMIs that have no corresponding entry in amazon-ec2-amis.nix:

❯ aws ec2 describe-images \
    --region us-west-2 \
    --filters Name=owner-id,Values=080433136561 \
  | jq '.Images | map(select(.Architecture == "arm64")) | sort_by(.CreationDate) | reverse | map({ ImageId, Description }) | .[0]'
{
  "ImageId": "ami-07dfadedb6493942b",
  "Description": "NixOS 23.05.555.52869451b83 aarch64-linux"
}

I propose that we tackle these issues with automation:

• We write a script to auto-generate amazon-ec2-amis.nix,
• Then instructions for the AMI generation are added to NixOS release playbook (I assume such a thing exists?)

[cyounkins]

@AmineChikhaoui

[Sleepful]

Right, I found ami-01ddbf9f89efa2eb7 by looking at the AMIs from the same owner (id 080433136561) as one of the offical AMIs that could be found sometime in the past in https://nixos.org/download.html#nixos-amazon

now thereof page has empty fields in the AWS AMI section and it can be really confusing for new users.

The AMI name I mentioned is NixOS-23.05.555.52869451b83-aarch64-linux and it does not show in the mentioned list https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/amazon-ec2-amis.nix

[endgame]

Hi, I wrote that blog post. I don't remember finding anything anywhere that said 080433136561 is the true source of official NixOS AMIs, though it does seem to be the case for the official ones to date. I think I inferred it by working backwards, looking up the owner id of an image that does exist in amazon-ec2-amis.nix:

$ aws --region us-east-1 ec2 describe-images --image-ids ami-0a061ca437b63df33 | jq '.Images[0].OwnerId'
"080433136561"

I recently noticed that some other account has started uploading 23.11 images, but since I have no way to confirm they're legit, there's no way I'm running them in my AWS account:

BTW, I've filed https://github.com/NixOS/nixos-homepage/issues/1325 for the busted home page. I have no idea what generates the page, so I can't PR it.

There's also a bunch of useful context at https://discourse.nixos.org/t/ami-for-nixos-23-11/36860:

@arianvp says:

The NixOS infra team created a new AWS account for AMIs (as we were uncomfortable giving me access to the account they’re currently uploaded in as that hosts also other stuff that I don’t want access to at all).

So I guess that explains the account switch. @arianvp: Can we get a public page that says which account has the official AMIs, so people know these are trustworthy? I can go back and update my blog once things are confirmed for 23.11 and beyond.

Also #292886 should close off this issue.

[endgame]

Also also: #292886 adds to the release notes a mention of a new AWS account and a link to https://nixos.github.io/amis which lists the latest ones, as well as promising the eventual removal of amazon-ec2-amis.nix:

https://github.com/NixOS/nixpkgs/pull/292886/files#diff-9c253a17d297f3da51e6ef73a3cb898d879ccf8b0c8613199770917e9387d92cR45-R49

[arianvp]

From what i understood from the nixos marketing team https://github.com/NixOS/nixpkgs/pull/292886 should also fix the homepage until we fix it to use the new data source.

[samuela]

Is there a change we could make to resolve this issue in future releases as well? Ideally we could avoid this whac-a-mole on each release.

[arianvp]

This is taken care of. Https://nixos.github.io/amis is automatically kept up to date through a GitHub action. There will be no need anymore for human intervention. It will even upload new AMIs for each channel bump within a release.

The idea is that the homepage will consume https://nixos.github.io/amis/images.json and the amazon-ec2-amis.nix will be deprecated or deleted. I'm in touch with them to make this change.

[arianvp]

By the way @endgame would you prefer if I change the naming convention to be the same as before? I can still make that change

[samuela]

The idea is that the homepage will consume https://nixos.github.io/amis/images.json and the amazon-ec2-amis.nix will be deprecated or deleted. I'm in touch with them to make this change.

Amazing news! Thanks so much @arianvp !

[endgame]

By the way @endgame would you prefer if I change the naming convention to be the same as before? I can still make that change

I would prefer it, but only mildly. It doesn't matter too much what the naming convention is, so long as it's consistent and we can prove that the images come from NixOS. If there are advantages to the new scheme, or it's too much work, I'm not going to lose sleep over it.

But when the naming convention and the owner ID changed, I became suspicious.