Fluentd plugin to add Amazon EC2 metadata fields to a event record
fluent-plugin-ec2-metadata | fluentd | ruby |
---|---|---|
>= 0.1.0 | v0.14.x | >= 2.1 |
0.0.15 <= | v0.12.x | >= 1.9 |
Use RubyGems:
gem install fluent-plugin-ec2-metadata
Example:
<match foo.**>
@type ec2_metadata
aws_key_id YOUR_AWS_KEY_ID
aws_sec_key YOUR_AWS_SECRET/KEY
metadata_refresh_seconds 300 # Optional, default 300 seconds
imdsv2 true # Optional, default false
output_tag ${instance_id}.${tag}
<record>
hostname ${tagset_name}
instance_id ${instance_id}
instance_type ${instance_type}
az ${availability_zone}
private_ip ${private_ip}
vpc_id ${vpc_id}
ami_id ${image_id}
account_id ${account_id}
</record>
</match>
Assume following input is coming:
foo.bar {"message":"hello ec2!"}
then output becomes as below (indented):
i-28b5ee77.foo.bar {
"hostname" : "web0001",
"instance_id" : "i-28b5ee77",
"instance_type" : "m1.large",
"az" : "us-west-1b",
"private_ip : "10.21.34.200",
"vpc_id" : "vpc-25dab194",
"account_id" : "123456789",
"image_id" : "ami-123456",
"message" : "hello ec2!"
}
Or you can use filter version:
<filter foo.**>
@type ec2_metadata
aws_key_id YOUR_AWS_KEY_ID
aws_sec_key YOUR_AWS_SECRET/KEY
metadata_refresh_seconds 300 # Optional, default 300 seconds
imdsv2 true # Optional, default false
<record>
hostname ${tagset_name}
instance_id ${instance_id}
instance_type ${instance_type}
private_ip ${private_ip}
az ${availability_zone}
vpc_id ${vpc_id}
ami_id ${image_id}
account_id ${account_id}
</record>
</filter>
The following placeholders are always available:
${tag_parts[0]}
or ${tag_parts[-1]}
The followings are available when you define aws_key_id
and aws_sec_key
(or define IAM Policy):
The following is an example for a minimal IAM policy needed to ReadOnlyAccess to EC2.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "ec2:Describe*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "elasticloadbalancing:Describe*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"cloudwatch:ListMetrics",
"cloudwatch:GetMetricStatistics",
"cloudwatch:Describe*"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "autoscaling:Describe*",
"Resource": "*"
}
]
}
Refer to the AWS documentation for example policies. Using IAM roles with a properly configured IAM policy are preferred over embedding access keys on EC2 instances.
git checkout -b my-new-feature
)git commit -am 'Add some feature'
)git push origin my-new-feature
)