Simple Feature Request: Retrieve & Set Hostname

[armenr]

As per this thread: https://community.clearlinux.org/t/how-to-retrieve-instance-hostname-from-ucd/8096

Context

I know that ucd generates /var/lib/cloud/aws-user-data when the ucd systemd oneshot runs.

What I want to understand is whether ucd also fetches the instance hostname from metadata → the same output that you’d get from curl --silent http://169.254.169.254/latest/meta-data/hostname on an EC2.

The reason I ask is because putting curl --silent http://169.254.169.254/latest/meta-data/hostname > /opt/hostname.txt into a systemd oneshot appears to take about 5 seconds to run, and ucd is significantly faster (and has already done the work of interacting with the metadata API).

Even a 5 second boot delay here has a practical/measurable impact on our infrastructure use-case...although I know I might be sounding greedy or a bit too "optimize-all-the-things!"

The Feature

Enable UCD to set a cloud instance's default hostname based on the user metadata API (hostnamectl + /etc/hosts/).

^^ IF the above is too opinionated or controversial, could we at least get the hostname dropped off/saved somewhere in /var/lib/cloud/ ?

Thank you!! :)

[ahkok]

@bwarden and me discussed this and it wouldn't be too difficult to add*:

• We can extend the ucd-data-fetch to do an additional http request to fetch the hostname file if it is known that the data provider supports this (i.e. it has to be a simple file with just the name in it, on a separate simple URL on the server)
• It can then expand the written out user-data file to add this name.

[armenr]

This would be much appreciated, and very useful! Thank you for the quick response and sharing your ideas.

I have one more question to ask:

The "initial" request to the AWS Metadata API appears to average around ~5 seconds (no matter how I try it).

I'm curious...With ucd or ucd-data-fetch --> How were you able to get that time down to ~1 second?

For example, in my testing:

On a completely "new"/"cold" EC2 instance...the first call to IMDS (metadata service) takes about 5 seconds (using curl or wget). ALL subsequent requests to IMDS are extremely fast...but not the first one.

How does micro-config-drive manage to retrieve the metadata so quickly?

[ahkok]

I'm curious...With |ucd| or |ucd-data-fetch| --> How were you able to get that time down to ~1 second?

Message ID: @.***>

ucd-data-fetch fetches with HTTP only, and directly connects to a hard-coded IP address, so there is no hostname lookup, TLS/SSL or DNS required. This can save a ton of time on a freshly booted system, as all those caches are empty right after boot.

[armenr]

@ahkok Thank you for that explanation and response. I'm not going to lie...I haven't looked at or written and C in a very, very long time, so forgive my ignorance when I ask:

Does the http client implementation also take IMDS v2 into consideration? It's become common practice to deprecate/disable IMDS v1 access on newly-launched EC2s.

[bwarden]

Does the http client implementation also take IMDS v2 into consideration?

No, the client fetcher only does simple GET requests -- no tokens.

[armenr]

@bwarden - Thanks for the clarification.

The typical/common practice these days (the prescribed "best-practices" from AWS) is to lock down your EC2 instance Metadata configuration to IMDSv2 (which requires tokens) - but does not require SSL.

I wonder/worry whether a lack of IMDSv2 support would be prohibitive to ClearLinux adoption.

Thanks again for the quick turn around on the feature request, also!

[armenr]

@bwarden @ahkok - With this now implemented, how can I use ucd to explicitly set the retrieved hostname at boot time?

I'm really sorry to be a nag...I'm just a recent adopter (and AVID lover) of ClearLinux.

I understand that /var/lib/cloud is the place to drop off the cloud-init scripts (right?), but how would I tell it to set the hostname from what is retrieved at initialization (when ucd runs).

This one example would be really, really helpful. Thank you (again)! :)

[armenr]

So I was able to test and ensure my userdata is being passed to ucd, however I don't see anything for host or hostname

root@clr-ec299ecb4ed1ce941d47061a097d05ad /var/lib/cloud # cat aws-user-data 
#cloud-config
users:
  - name: clear
    groups: wheelnopw
ssh_authorized_keys:
  - ssh-rsa <REDACTED>
#cloud-config
write_files:
  -
    content: |
        A test file
    path: /tmp/cloud-init-test-1
    owner: nobody.nogroup
    permissions: 0644
  -
    content: |
        Another test file
    path: /tmp/cloud-init-test-2
    owner: root.root
    permissions: 0600

[bwarden]

Do you actually get a value from curl --silent http://169.254.169.254/latest/meta-data/hostname? If the fetcher doesn't get anything, it won't write the hostname: tag. If it does, it will write it immediately following the SSH keys and prior to the user data.

This is our expected test output when we run a self-hosted server: https://github.com/clearlinux/micro-config-drive/blob/master/tests/fetch_data/expected

[ahkok]

I don't think this feature has made it into a release of clearlinux just yet.Message ID: @.***>

[bwarden]

Looks like it finally landed in release 37610.

[armenr]

Thank you for the updates @ahkok & @bwarden, and for the really fast and great work.

Unfortunately, this caused me to discover a different, AWS-specific bug --

If an EC2 instance doesn't possess an SSH key, the run fails/breaks. Unfortunately, as a security best-practice (and as a method of reducing risk/attack surface), we do not provision SSH keys with/for EC2s, at all. We follow the AWS best-practice of only granting assumed access through the AWS SSM agent.

😢 but I think that might be a separate Github Issue...

[bwarden]

but I think that might be a separate Github Issue...

Yes, please open a new issue for that. Include a redacted copy of your aws-cloud-data file if you can.