Early error when using on a Scaleway IPV6 instance

[chpill]

Hello, thank you for this very promising tool!

I ran into an error while using it on a Scaleway DEV1-S instance without and IPV4 address, only IPV6 (I tested with Ubuntu 22.04 Jammy Jellyfish and Debian Bookworm base images):

$ nix run github:numtide/nixos-anywhere -- --flake .#chpill-test root@2001:bc8:610:e803::1

### Uploading install SSH keys ###
/run/current-system/sw/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/tmp/tmp.D7PT7M2N3G/nixos-anywhere.pub"
/run/current-system/sw/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/run/current-system/sw/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
Warning: Permanently added '2001:bc8:610:e803::1' (ED25519) to the list of known hosts.

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh -o 'ConnectTimeout=10' -o 'UserKnownHostsFile=/dev/null' -o 'StrictHostKeyChecking=no' 'root@2001:bc8:610:e803::1'"
and check to make sure that only the key(s) you wanted were added.

### Gathering machine facts ###
Warning: Permanently added '2001:bc8:610:e803::1' (ED25519) to the list of known hosts.
### Switching system into kexec ###
Warning: Permanently added '2001:bc8:610:e803::1' (ED25519) to the list of known hosts.
Warning: Permanently added '2001:bc8:610:e803::1' (ED25519) to the list of known hosts.
curl: (28) Failed to connect to github.com port 443 after 130879 ms: Connection timed out

gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now

This happened to me while using https://github.com/numtide/nixos-anywhere-examples/.

Looking at the bash script, I notice that it tries to download an archive from Github, and that seems to fail. When I log into the instance, indeed the following command does not work: curl -I https://github.com/nix-community/nixos-images/releases/download/nixos-23.05/nixos-kexec-installer-noninteractive-x86_64-linux.tar.gz

As soon as I attach an IPV4 address to the instance (using the Scaleway console in my case), the previous curl gives an answer, and re-runing nixos-anywhere seems to work as expected.

I am not sure if I made a mistake somewhere or if Github prevents IPV6 hosts to access their downloads in general.

[mibmo]

Could it be that you're not setting up IPv6 correctly? I was doing this exact thing yesterday (nice timing! :)) and I had to accept router advertisements

systemd.network = {
  enable = true;
  networks."lan-10" = {
    matchConfig.Name = "eno1";
    networkConfig = {
      DHCP = "no";
      IPv6AcceptRA = true;
    };
  };
};

networking.dhcpcd.enable = true;

Try curl -6 btw. It'll force curl to use ipv6.

EDIT: wow! Github doesn't actually have IPv6 support - that is .... strange, in 2023.

[mibmo]

Also I don't think it actually matters, but it's nix-community/nixos-anywhere, not numtide/nixos-anywhere now :)

[chpill]

@mibmo thanks, I'm very new to the IPV6 stuff, so it's likely I'm not configuring stuff properly. I'll go read up on router advertisement.

But I'm not sure I understood your comment, did you actually get nixos-anywhere to work in a IPV6 only setup? It seems to me that the failure occurs before the nixos system is configured.

[mibmo]

@chpill oh yeah it seems I had an ipv4 address while I was setting up, I guess it used that.

You'll have to use a workaround right now, as Github still doesn't support IPv6. Your options are essentially a. use a transition layer like nat64; see networking.jool b. use a proxy like github.via-ipv6.com (not a very good option imo) c. clone the repos you'll need to use, host them on an ipv6-compatible host (like gitlab or bitbucket) and patch everything d. pay up and use ipv4 :(

Is your reasoning for being ipv6-only purely financial or is there something else? (I know Scaleway's 3EUR/month is kinda hefty if you have a bunch Picos or something)

[chpill]

@mibmo well, there's a financial aspect to it, but the extra IPV4 is also one more thing to provision and take care of. I want to learn how to manage infrastructure with terraform openTofu and the scaleway plugin, so I'm trying to sort out what is necessary and what is not. I have also heard that an IPV6 address was more secure because it was far less discoverable that an IPV4 one. Being new to the sysadmin game, I'm not sure I can properly secure a device that is easily discoverable. I'll look into the nat64 thing you mentionned, thanks again!

Coming back to this project though, I guess we'd all agree that we would rather have nixos-anywhere than nixos-anywhere-ipv4. Couldn't it work if the nixos-kexec-installer-noninteractive-x86_64-linux.tar.gz was hosted somewhere IPV6 compatible?

[mibmo]

I want to learn how to manage infrastructure with terraform openTofu and the scaleway plugin, so I'm trying to sort out what is necessary and what is not.

Nice! I wish you luck :)

I have also heard that an IPV6 address was more secure because it was far less discoverable that an IPV4 one

This is security through obscurity and you really shouldn't consider this "extra security"; it's harder to scan an IPv6 range than an IPv4 one (due to there usually being more addresses), but if your server is already public facing then it's trivial to get the IPv{4,6}.

Couldn't it work if the nixos-kexec-installer-noninteractive-x86_64-linux.tar.gz was hosted somewhere IPV6 compatible?

Maybe? It's far from the only you'll need to fetch from GitHub when using NixOS so either way you'll need to access GitHub somehow.

[phaer]

Couldn't it work if the nixos-kexec-installer-noninteractive-x86_64-linux.tar.gz was hosted somewhere IPV6 compatible?

Yes, that would solve your problem at hand; the root cause is that github.com still doesn't support ipv6. There's no public mirror that i know of, but you can host the image yourself and pass --kexec-url to nixos-anywhere.

[zimbatm]

You can now use the https://gh-v6.com service as a proxy for the images. So for example:

nixos-anywhere \
  --kexec https://gh-v6.com/nix-community/nixos-images/releases/download/nixos-unstable/nixos-kexec-installer-noninteractive-x86_64-linux.tar.gz \
  ...

[Mic92]

Documented how to use nixos-anywhere with ipv6: https://github.com/nix-community/nixos-anywhere/pull/244