Add support for EC2 instance metadata.

michaelbaamonde commented 9 years ago

This adds support for exposing EC2 instance metadata as facts. It determines whether the host is running on EC2 by attempting a GET request against the metadata endpoint (http://169.254.169.254/latest/meta-data). I've set the timeout for this attempt at 50 millieconds. I ran terminus well over a hundred times on an instance and never found it to take longer than this. It does, of course, add an additional 50 milliseconds of latency for machines not running on EC2, but this borders on the imperceptible.

Example output of the EC2 facts:

{
  "EC2": {
     "AmiID": "ami-ae62d7c6",
     "AmiLaunchIndex": 0,
     "AmiManifestPath": "ubuntu-us-east-1/images/ubuntu-precise-12.04-amd64-server-20140924.4.manifest.xml",
     "AvailabilityZone": "us-east-1d",
     "Hostname": "ip-12-345-678-901.ec2.internal",
     "InstanceAction": "none",
     "InstanceID": "i-01234ab5",
     "InstanceType": "m3.medium",
     "KernelID": "aki-123abcd4",
     "LocalHostname": "ip-12-345-678-901.ec2.internal",
     "LocalIPV4": "12.345.6781.901",
     "MAC": "22:00:0B:10:04:DE",
     "Profile": "default-paravirtual",
     "PublicHostname": "ec2-54-123-456-789.compute-1.amazonaws.com",
     "PublicIPV4": "54.123.456.789",
     "ReservationID": "r-d0d001ab",
     "SecurityGroups": [
       "foo",
       "default"
     ]
   }
}

I added a TODO to add additional types of metadata (e.g. block device mappings and IAM-related metadata), and would like to tackle these soon.

Thanks!

kelseyhightower commented 9 years ago

@michaelbaamonde I wonder if we should start a plugins directory and add this fact there? I'm thinking that terminus core facts should stay small and things like this live as executable facts under /etc/terminus/facts.d. To ease installation we can build this and cut official releases on this repo. What are your thoughts on this?

lwf commented 9 years ago

I have also implemented a standalone fact for retrieving EC2 info: https://github.com/lwf/terminus-facts/blob/master/ec2/main.go. While the code is not particularly idiomatic it does retrieve all information from the meta-data API recursively. I'd generally prefer this to having a bunch of hand-picked facts, but YMMV.

michaelbaamonde commented 9 years ago

@kelseyhightower:

Yeah, that's a good point and one I wanted to let you chime in on before making too many assumptions. Comparable projects like facter and ohai take the plugin approach, and I agree that it's probably the way to go, especially as additional infrastructure providers are added.

My only concern (which you seem to have anticipated) is the build/release process for "official" plugins maintained in this repo. If you do indeed want to maintain plugins written in Go in this repository directly, it seems like it'd be a little cumbersome to require that they be separate executables, as they'd wind up repeating lots of code and probably entail a performance hit, as well.

Perhaps we could have an additional code-loading mechanism for such things. It really just turns on whether you'd like to have a way to build plugins into the terminus binary directly. If you want to do that, I'd be happy to help implement it and alter this patch accordingly. If not, I'm happy to convert this to a standalone fact, move it under a plugins/ subdirectory, and help out with cutting releases.

@lwf:

The recursive approach is cool. I initially was going to do something similar, but I tried to conform to the conventions for keys (i.e. camel case) and such in the JSON output, which makes a generic "get-all-the-metadata" function a little trickier. I wanted to get a conversation about it going, so I punted for the time being.

That said, your plugin slows things down considerably-- perhaps consider using goroutines to speed it up?

kelseyhightower / terminus

Add support for EC2 instance metadata. #5