search

hashicorp / nomad

Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.
https://www.nomadproject.io/
Other
14.78k stars 1.94k forks source link

Add AWS instance tags to client metadata #12537

Open nahsi opened 2 years ago

nahsi commented 2 years ago

Since January 2022 AWS Instance tags are available in EC2 metadata. Would it be possible to add Instance tags to Nomad client metadata?

Tags in metadata must be enabled first.

Then tags can be accessed from metadata endpoint:

nahsi@i-0f8c34d628a7cae51:~$ curl http://169.254.169.254/latest/meta-data/tags/instance
Name
aws:autoscaling:groupName
aws:ec2:fleet-id
aws:ec2launchtemplate:id
aws:ec2launchtemplate:version
class
env
role
terraform_managed

Proposal

Pupulate Nomad client metadata with Instance tags collected from AWS metadata endpoint

Use-cases

Ability to constraint jobs based on AWS tags.

DerekStrickland commented 2 years ago

Hi @nahsi

It seems like it should be possible. The AWS SDK mod might need to be updated. It looks like we are using v1.42.27 which came out Jan 4, so I don't know if it includes the new functionality or not. I suspect the changes would need to be made in the AWS fingerprinter. Do you feel like taking a pass at it? Pull requests are always welcome 😄

nahsi commented 2 years ago

@DerekStrickland I will try!

DerekStrickland commented 2 years ago

That's fantastic! Thanks for being willing. I'll look for the incoming PR.

mr-karan commented 1 year ago

@nahsi Are you still working on this? If not, I'd like to send a PR for the same. :smile:

nahsi commented 1 year ago

@nahsi Are you still working on this? If not, I'd like to send a PR for the same. :smile:

Hi. Not working on this 😅

EugenKon commented 5 months ago

@mr-karan Have you implemented PR for this?

EugenKon commented 5 months ago

Not sure, but it look like it is related: https://github.com/hashicorp/nomad/issues/6744

I also found this question: https://discuss.hashicorp.com/t/list-out-all-platform-variables/1731/7

Probably curl http://169.254.169.254/latest/meta-data/tags/instance/Name could be aliased to $unique.platform.aws.tags.Name, eg. when I refer this name a call to the AWS API is made.

Usage example:

job "exec-job" {
  region = "planitar"
  datacenters = ["dc1"]
  type = "sysbatch"

  group "exec" {
    task "exec-task" {
      driver = "exec"

      config {
        command = "/bin/bash"
        args    = ["local/exec.sh"]
      }

    constraint {
      attribute = "${meta.aws.tag.Name}"
      operator  = "equal"
      value     = "worker"
    }

      template {
        destination = "local/exec.sh"
        data        = <<-EOH
          set -ex
          echo "Hello ${meta.aws.tag.Name}" # As was proposed by ChatGPT )
        EOH
      }
    }
  }
}