search

coreos / fedora-coreos-tracker

Issue tracker for Fedora CoreOS
https://fedoraproject.org/coreos/
262 stars 59 forks source link

Uploading to cloud platforms: Packet #150

Open dustymabe opened 5 years ago

dustymabe commented 5 years ago

This is part of https://github.com/coreos/fedora-coreos-tracker/issues/146 and tracks the work/discussion around uploading to Packet.

vielmetti commented 5 years ago

One note to drop in re PXE, specifically iPXE - we are tracking this issue ipxe/ipxe#90 which yields intermittent failures with iPXE as a result of a bug in OCSP certificate validation. Any eyes on this especially for test and validation cycles would be helpful.

vielmetti commented 5 years ago

See also https://github.com/coreos/fedora-coreos-tracker/issues/135

bgilbert commented 5 years ago

I expect our approach will look similar to CL:

vielmetti commented 5 years ago

As noted on the 7/24/2019 IRC meeting channel, two current blocking issues are #24 (network management) and #105 (live IPXE issues).

bgilbert commented 4 years ago

FCOS works on Packet today using the custom_ipxe image type and the following iPXE script:

#!ipxe

set version 31.20200310.2.0
set base-url https://builds.coreos.fedoraproject.org/prod/streams/testing/builds/${version}/x86_64
kernel ${base-url}/fedora-coreos-${version}-live-kernel-x86_64 ip=dhcp rd.neednet=1 initrd=fedora-coreos-${version}-live-initramfs.x86_64.img console=ttyS1,115200n8 coreos.inst.install_dev=/dev/sda coreos.inst.stream=testing coreos.inst.ignition_url=<URL-TO-IGNITION-CONFIG> coreos.inst.platform_id=packet
initrd ${base-url}/fedora-coreos-${version}-live-initramfs.x86_64.img
boot

Note that the above installs current FCOS testing to disk and reboots into it. It should also be possible to drop the install kargs and run directly from RAM.

Because we don't have support for Packet-specific networking, the machine only gets a public IPv4 address, but it does install and apply the Ignition config and SSH keys.

Because the install flow involves coreos-installer, and the installer supports overriding the platform ID, it may not even make sense to publish a distinct image for Packet. The Packet folks might have an opinion on that, though.

vielmetti commented 4 years ago

@bgilbert -

What is the brief state of #24 (network management) ? It should be possible to configure the host from the metadata service we provide, which is sufficiently similar to the AWS EC2 setup that it should be only a little bit of code to implement. That would give you access to information about the private IPv4 address and also IPv6 networking.

I would love to see an image packaged in such a way that someone could pick FCOS from a Packet menu and be up and running without any more install work than necessary. Looks like we are close to that with the iPXE script.

bgilbert commented 4 years ago

We're working on switching initramfs networking to NetworkManager (https://github.com/coreos/fedora-coreos-tracker/issues/394). Afterburn knows how to talk to the Packet metadata service, but it only knows how to write systemd-networkd configs and would need to be updated for NM. Alternatively, it might make sense to teach NM how to talk to the metadata service directly.

sufficiently similar to the AWS EC2 setup that it should be only a little bit of code to implement.

Hmm, any pointers for that? On EC2 we just DHCP, I believe.

vielmetti commented 4 years ago

@bgilbert - ah, I was commenting more that the delta between the Packet code to talk to metadata and the EC2 to talk to metadata would be small; did not realize when I wrote this that NM did not speak to the metadata at all.

bgilbert commented 4 years ago

The bare-metal image currently configures a serial console on ttyS0, but Packet needs it on ttyS1. We could work around this with https://github.com/coreos/coreos-installer/issues/13 but it'd be better to automatically switch kernel arguments by platform.

jlebon commented 4 years ago

We could work around this with coreos/coreos-installer#13 but it'd be better to automatically switch kernel arguments by platform.

How do you imagine this working? Would the installer automatically add it based on whether the specified Ignition platform ID was packet?

bgilbert commented 2 years ago

xref https://github.com/coreos/fedora-coreos-tracker/issues/110#issuecomment-905811483 for platform-specific serial console kargs.