search

aws / aws-toolkit-azure-devops

AWS Toolkit for Azure DevOps
Other
235 stars 100 forks source link

Incompatible with IMDSv2 #531

Closed daiyyr closed 5 months ago

daiyyr commented 9 months ago

Describe the bug

CloudFormationCreateOrUpdateStack gets "...error: failed to query EC2 instance metadata for region - Error: null" in AL2023 agent

Please update the below function. Other we have to enforce all our AL2023 instances to restore IMDSv1. https://github.com/aws/aws-toolkit-azure-devops/blob/b0b549eeed155a296a686b3a955a8b03f7c08adb/src/lib/awsConnectionParameters.ts#L282

To reproduce

use AL2023 as a selfhosted agent

run task CloudFormationCreateOrUpdateStack in the agent

Expected behavior

run as nomral

Screenshots

Your Environment

Additional context In AL2023, you can't directly read data from http://169.254.169.254/latest/meta-data you need to get token first: TOKEN=$(curl --request PUT "http://169.254.169.254/latest/api/token" --header "X-aws-ec2-metadata-token-ttl-seconds: 3600") REGION=$(curl -s http://169.254.169.254/latest/meta-data/placement/region --header "X-aws-ec2-metadata-token: $TOKEN")

that's why the task fails in AL2023

rknechtel commented 6 months ago

I'm running into an issue that I am sure has to do with the lack of IMDSv2 support in the AWS Toolkit. I updated all my EC2 instances to IMDSv2 and suddenly all my S3 copies from S3 to my EC2 instances (using SSM) in my Azure DevOps pipelines stopped working. I contacted AWS Support but they have been no help. I asked them about when a new version of the AWS Toolkit would be released as it's been almost 2 years since the last release. I mentioned this specific PR to them. They knew nothing about it and referred me here.

After Several communications with AWS they have confirmed with the lack of IMDSv2 support in the AWS Toolkit is my issue. But they don't know when it will be fixed, nor when a new version will be released. Since it's been almost 2 years since they released a new version of this tool, it is probably safe to figure they won't be. I think AWS has abandoned this project.

If you do not NEED IMDSv2 support - DO NOT enable it if you are using Azure DevOps and the AWS Toolkit to deploy to EC2 instances or your pipelines will be forever dead.

I think to get any further updates to this project it will need to be forked.

rknechtel commented 5 months ago

After further discussions with AWS Support - they are now working on the PR for this. :)

Per AWS Support: After further discussion with the Toolkit team, it was agreed upon that there is unfortunately no good workaround for getting the system to work until the support for IMDSv2 is enabled for Azure DevOps. We will have to wait until the pr is merged and the new Toolkit is released.

Our Toolkit team is actively working on reviewing the pr: https://github.com/aws/aws-toolkit-azure-devops/pull/535 and an engineer was assigned to it yesterday. All communication regarding the pr will directly happen on the pr itself, so my recommendation is to keep an eye on it for all updates. Once the pr is merged, the toolkit release will be scheduled as soon as possible. If the new toolkit release to the marketplace does not fix the issue, the team would be happy to meet with you and setup a troubleshooting session.

daiyyr commented 5 months ago

Thank you @rknechtel !

rknechtel commented 5 months ago

Update from AWS on the PR.

"I have received word from the Toolkit team that they have addressed the blocking issue from this pr: https://github.com/aws/aws-toolkit-azure-devops/issues/522. They have tested the changes in https://github.com/aws/aws-toolkit-azure-devops/pull/535 and have confirmed that they now work properly in the Toolkit. I am currently waiting on more details regarding the release! "

rbbarad commented 5 months ago

Thank you for your patience with this issue. We've been addressing some blocking issues that have been preventing a release - these have now been fixed, and we are planning to release a toolkit update with these fixes this week. I will update this issue with more details as we have them.

rbbarad commented 5 months ago

Version 1.14.0 of the AWS Toolkit containing the fix for this issue has been released. Please let us know if you are experiencing any issues with this new build!

rknechtel commented 5 months ago

Thank you for your help!

On Wed, Jan 31, 2024, 5:13 PM Rishi Barad @.***> wrote:

Version 1.14.0 of the AWS Toolkit containing the fix for this issue has been released. Please let us know if you are experiencing any issues with this new build!

— Reply to this email directly, view it on GitHub https://github.com/aws/aws-toolkit-azure-devops/issues/531#issuecomment-1920230502, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANHOPO7F5AREPS6Q6Z4XTB3YRLM2DAVCNFSM6AAAAAA5CKIFESVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRQGIZTANJQGI . You are receiving this because you were mentioned.Message ID: @.***>

rknechtel commented 5 months ago

I just retested my full pipelines and they are working great again. Thank you for fixing this.