Add support for IMDSv2 on EC2 instances

[brantg]

When using NiFi (which uses Spring Vault 3.1.0 under the hood) on an EC2 instance with IMDSv2 set to 'Required', we get the error message "Cannot obtain Identity Document from http://169.254.169.254/latest/dynamic/instance-identity/document".

If we toggle IMDSv2 on the EC2 instance to 'Optional', the error does not appear.

Reviewing the latest version of Spring Vault, it seems to me that this would still be the case with 3.1.1.

We raised the issue with NiFi and they advised us to raise it instead as an issue with Spring Vault.

[mp911de]

We don't pass on any session headers to the metadata service. To support this, the following requirements would have to be defined:

• Who creates a session token
• How is the token provided to the auth mechanism?
• Who maintains TTL and how is a new token created/provided?

In a JWT/Kubernetes environment, the platform provides such details and ensures recent information stored in a file. Spring Vault is by no means authoritative of creating and maintaining yet another set of tokens so I suggest that you create your own ClientAuthentication implementation, potentially based on AwsEc2Authentication to explore provisioning and runtime operations.

With sufficient details from a broader community we can work on a path to enable session-bound usage of the identity service.

[brantg]

I don't think I understand the issue here - all you need to do is fetch the token from the EC2 instance itself, feed it back to the metadata call and then drop it. The token does not need to be kept beyond the call. ...or else I have greatly misunderstood how much Spring Vault needs to interact with the IMDS.

Documentation: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-metadata-v2-how-it-works.html

Specific CLI example:

TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`
curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/

[mp911de]

Thanks for the additional detail. My initial understanding was that AWS introduced a mechanism to secure metadata requests so that only the first process could request a session token and that no other process could obtain a new token until the other one expires.

After testing the new API a bit, I must say that we only gain additional overhead without getting actual value out of the process.

It should be possible to conditionally obtain a short-lived session token during the login.

[brantg]

Oh, completely agree about the zero value add... but it is what we are required to work with, so here we are.