[RFE] Support Azure Userdata for coreos-cloudinit

[pothos]

Current situation

Currently on Azure the Custom Data attribute is the location for Ignition/coreos-cloudinit configs. The (newer) VM Userdata attribute is ignored.

Impact

For users this can be confusing if they put the config into Userdata and nothing works. The Custom Data attribute is quite limited because it can't be updated and is only accessible on the very first boot which can cause problems, e.g., with the upcoming Ignition v3 kargs directive that issues a reboot and then wants Ignition to run again.

Ideal future situation

The Userdata attribute is supported and the recommended way of passing configs.

Implementation options

Check if Ignition upstream already supports it (otherwise implement it upstream) - we will get the support together with the planned Ignition v3 update. Implement it for coreos-cloudinit.

Additional information

[pothos]

Checked Ignition support and only Custom Data (the emulated CD-ROM) is supported at the moment: https://github.com/coreos/ignition/blob/main/internal/providers/azure/azure.go

[tormath1]

Checked Ignition support and only Custom Data (the emulated CD-ROM) is supported at the moment: https://github.com/coreos/ignition/blob/main/internal/providers/azure/azure.go

Yeah. I'm currently looking at the Azure SDK to get something similar to AWS: https://github.com/coreos/ignition/blob/main/internal/providers/aws/aws.go#L123-L139.

Resources:

• https://docs.microsoft.com/en-us/azure/virtual-machines/windows/instance-metadata-service?tabs=linux#get-user-data

EDIT: I think the logic could be the following:

1. Try to fetch user-data through: http://169.254.169.254/metadata/instance/compute/userData endpoint using FetchToBuffer
2. Fallback on custom-data if it fails

Current user-data on Azure seems to be equivalent to AWS IMDSv1 format.

[bgilbert]

The Custom Data attribute [...] is only accessible on the very first boot which can cause problems, e.g., with the upcoming Ignition v3 kargs directive that issues a reboot and then wants Ignition to run again.

As I recall, custom data is accessible until the VM reports to Azure that it is ready. If the Afterburn Azure checkin is run After=ignition-kargs.service in the initrd, or in the real root, kargs should work properly with custom data.

[jepio]

Indeed the custom data dvd is available until it is either ejected or the check-in happens. On check-in the host will hot-unplug the whole dvd drive.

We don't use afterburn on Azure, since we ship the Azure Linux Agent and so the checkin happens at the end of boot, not in initrd.

[pothos]

So, Ignition is done but coreos-cloudinit not - I guess that's ok for now and when we updated Ignition to include the feature we can already call it out in the docs.

[tormath1]

ignition-2.14.0 has been merged including the Azure IMDS userData feature - it should be available in the next Alpha release. We still have this PR remaining PR https://github.com/flatcar-linux/mantle/pull/321 to enable tests. :)