New GCE NixOS images can't be listed

[asymmetric]

Describe the bug Since the switch away from the nixos-cloud-images bucket for NixOS images, the images can't be listed anymore:

To Reproduce Steps to reproduce the behavior:

❯ gcloud compute images list --uri --project nixos-cloud --no-standard-images
ERROR: (gcloud.compute.images.list) Some requests did not succeed:
 - Required 'compute.images.list' permission for 'projects/nixos-cloud'

❯ gsutil ls -l gs://nixos-cloud
BucketNotFoundException: 404 gs://nixos-cloud bucket does not exist.

Expected behavior The gcloud command above should return a list of images, namely (at the moment) 20.09.

Additional context 20.09 is still available on the nixos-cloud-images bucket:

❯ gsutil ls -l gs://nixos-cloud-images
 291081495  2015-02-24T08:00:47Z  gs://nixos-cloud-images/nixos-14.12.323.91643074-x86_64-linux.raw.tar.gz
 287705475  2015-02-27T10:33:45Z  gs://nixos-cloud-images/nixos-14.12.471.1f09b77-x86_64-linux.raw.tar.gz
 311234775  2015-09-18T19:41:02Z  gs://nixos-cloud-images/nixos-15.09.406.5cd5fe3-x86_64-linux.raw.tar.gz
 311387788  2015-09-23T08:16:28Z  gs://nixos-cloud-images/nixos-15.09.425.7870f20-x86_64-linux.raw.tar.gz
 397747554  2020-05-03T20:14:47Z  gs://nixos-cloud-images/nixos-20-03-1639-73e73c7d6b5.raw.tar.gz
 397547630  2020-05-06T23:51:28Z  gs://nixos-cloud-images/nixos-20-03-1757-404fef90dee.raw.tar.gz
 300627279  2016-05-25T09:43:01Z  gs://nixos-cloud-images/nixos-image-16.03.847.8688c17-x86_64-linux.raw.tar.gz
 219901551  2017-05-02T12:53:43Z  gs://nixos-cloud-images/nixos-image-17.03.1082.4aab5c5798-x86_64-linux.raw.tar.gz
 262259697  2018-04-06T08:33:24Z  gs://nixos-cloud-images/nixos-image-18.03.131802.4b4bbce199d-x86_64-linux.raw.tar.gz
 262216695  2018-04-20T08:51:05Z  gs://nixos-cloud-images/nixos-image-18.03.132021.c0c5571ec1a-x86_64-linux.raw.tar.gz
 314671625  2018-05-30T17:32:19Z  gs://nixos-cloud-images/nixos-image-18.03.132536.fdb5ba4cdf9-x86_64-linux.raw.tar.gz
 261463897  2018-04-06T08:52:03Z  gs://nixos-cloud-images/nixos-image-18.03.git.207b429-x86_64-linux.raw.tar.gz
 344473043  2018-11-16T09:16:51Z  gs://nixos-cloud-images/nixos-image-18.09.1228.a4c4cbb613c-x86_64-linux.raw.tar.gz
 343988660  2018-11-22T19:22:50Z  gs://nixos-cloud-images/nixos-image-18.09.git.51076ed-x86_64-linux.raw.tar.gz
 398897148  2021-02-09T15:36:41Z  gs://nixos-cloud-images/nixos-image-20.03.3324.929768261a3-x86_64-linux.raw.tar.gz
 428773598  2021-03-19T15:38:07Z  gs://nixos-cloud-images/nixos-image-20.09.3505.12d9950bf47-x86_64-linux.raw.tar.gz
 427319071  2021-03-19T22:08:44Z  gs://nixos-cloud-images/nixos-image-20.09.3531.3858fbc08e6-x86_64-linux.raw.tar.gz
 399916806  2021-02-09T15:56:12Z  gs://nixos-cloud-images/nixos-image-21.03.git.6bf223c82e0-x86_64-linux.raw.tar.gz
TOTAL: 18 objects, 5961213787 bytes (5.55 GiB)

But since this is apparently deprecated, I'm not relying on these images anymore.

Notify maintainers

cc @AmineChikhaoui

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:

[bryanasdev000]

I didn't know about this change, I was using (gs://nixos-images) for some time without problems. Interested in the output of this issue.

[asymmetric]

@bryanasdev000 for context, there are 3 places where images are stored:

• gs://nixos-images (mostly old ones)
• gs://nixos-cloud-images (apparently now deprecated)
• nixos-cloud (no bucket here)

Seems like the switch nixos-images -> nixos-cloud-images happened in b894dd8b821 (3 years ago).

[bryanasdev000]

@bryanasdev000 for context, there are 3 places where images are stored:

• gs://nixos-images (mostly old ones)
• gs://nixos-cloud-images (apparently now deprecated)
• nixos-cloud (no bucket here)

Seems like the switch nixos-images -> nixos-cloud-images happened in b894dd8 (3 years ago).

Makes sense, but someone uploaded a 20.03 release, that's the one I am using for quite some time (since I found in nixos.wiki).

 397747554  2020-05-03T02:36:23Z  gs://nixos-images/google-cloud-nixos-20.03.1639.73e73c7d6b5.raw.tar.gz
 256556736  2014-12-17T10:51:00Z  gs://nixos-images/nixos-14.10pre-git-x86_64-linux.raw.tar.gz
 291081495  2015-01-16T16:36:46Z  gs://nixos-images/nixos-14.12.323.91643074-x86_64-linux.raw.tar.gz
 290985235  2014-12-19T12:45:58Z  gs://nixos-images/nixos-14.12.542.4c9ef9f7-x86_64-linux.raw.tar.gz
TOTAL: 4 objects, 1236371020 bytes (1.15 GiB)

[asymmetric]

Yeah, I think there should be a policy about refraining from manually pushing images IMO

[bryanasdev000]

Yeah, I think there should be a policy about refraining from manually pushing images IMO

Ideally, yes, this way we guarantee the reliability of the image, which ends up being an important step to guarantee NixOS in the main cloud providers.

[AmineChikhaoui]

@asymmetric We haven't switched from nixos-cloud-images bucket, see https://github.com/NixOS/nixpkgs/blob/master/nixos/maintainers/scripts/gce/create-gce.sh#L6 which is the script that creates the images. I researched a bit if we can allow compute.images.list to all authenticated users but so far I haven't found a way, I see in the documentation they mention:

Caution: Publicly shared images do not appear in the images list for users. Users must know the image name explicitly. Users cannot rely on getting the information by making an images.list request.

So maybe it's not even possible. If you find a way let me know and I can apply that, so far seems we can allow using the image given a name which is now hardcoded and updated in https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/gce-images.nix#L13. Listing the bucket can be used but likely not very reliable as there might be images created for testing for example. But I guess we can have a policy to not leave images unless it's officially added.

[bryanasdev000]

@asymmetric We haven't switched from nixos-cloud-images bucket, see https://github.com/NixOS/nixpkgs/blob/master/nixos/maintainers/scripts/gce/create-gce.sh#L6 which is the script that creates the images. I researched a bit if we can allow compute.images.list to all authenticated users but so far I haven't found a way, I see in the documentation they mention:

Caution: Publicly shared images do not appear in the images list for users. Users must know the image name explicitly. Users cannot rely on getting the information by making an images.list request.

So maybe it's not even possible. If you find a way let me know and I can apply that, so far seems we can allow using the image given a name which is now hardcoded and updated in https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/gce-images.nix#L13. Listing the bucket can be used but likely not very reliable as there might be images created for testing for example. But I guess we can have a policy to not leave images unless it's officially added.

I don't know how it works behind the scenes, but it would be interesting to have the images available for use equal to that of Debian for example, it would help a lot, both in the use and in the visualization of NixOS.

I'll see if I can find anything about it and comment again here.

[bryanasdev000]

So far I haven't found anything, just that certain images are only supported by the community, like Debian, but no information on how to submit a new image in this "format".

https://cloud.google.com/compute/docs/images#os-compute-support

https://cloud.google.com/compute/docs/images#lifecycle_policy

Maybe by contacting GCP or talking to someone from Debian we can get an idea of ​​what we need to do.

[stale[bot]]

I marked this as stale due to inactivity. → More info

[tylerjl]

Hey y'all, I know this is an old issue, but I don't intend to necro idly here and would be happy to help with the effort.

Lately I've been doing a lot of work with GCE images and found the same things noted here in addition to some other requirements like GCP marketplace base images being limited to "supported" images which it sounds like they're blessed in some way by GCP.

I infer that there's probably two paths forward here:

• Engage with Google/GCP somehow to learn what requirements, if any, there are to get NixOS endorsed as a supported public image. We're already part of the way there by having automation publish new images into GCP, so I would guess that maybe they have some other requirements like vuln scanning maybe? But that's only a guess.
• Engage with other distro maintainers like @bryanasdev000 noted to get an idea about the process. Debian and Ubuntu sound like reasonable places to start?

The easiest next steps would be if somebody has connections with maintainers in other distros that are in the supported OS list or connections at GCP to try and kickstart the conversations. I don't have either, so I'll wait a little while to see if anybody does, and if not, we can start a thread somewhere (mailing list, etc.) to at least learn about what the requirements are.