search

aws / amazon-ssm-agent

An agent to enable remote management of your EC2 instances, on-premises servers, or virtual machines (VMs).
https://aws.amazon.com/systems-manager/
Apache License 2.0
1.06k stars 324 forks source link

Snap in amazon-ssm-agent causing DevOps pipeline failures and uses outdated core18 by default #590

Closed nicanorflavier closed 3 weeks ago

nicanorflavier commented 1 month ago

Some of our DevOps pipelines failed security audits because the amazon-ssm-agent snap still uses the outdated core18 base. We expected core24 by now, and it's causing extra work.

Why hasn’t AWS updated this in all Ubuntu public AMIs? /snap/core18/2829/usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 /snap/core18/2829/usr/lib/x86_64-linux-gnu/libssl.so.1.1

There is no point in depending on an old and outdated core18 with known vulnerabilities, why not use a more current version like core24 which as updated security patches?

image001

jsf9k commented 1 month ago

I've found that the candidate channel actually corresponds to the distro packages version AWS offers for non-Ubuntu systems. The version in the candidate channel is currently 3.3.859.0 (from 2024-09-16) and it depends on core22 instead of core18.

nicanorflavier commented 1 month ago

I've found that the candidate channel actually corresponds to the distro packages version AWS offers for non-Ubuntu systems. The version in the candidate channel is currently 3.3.859.0 (from 2024-09-16) and it depends on core22 instead of core18.

Yeah I found out that one that you can remove the amazon-ssm-agent snap, then remove the core18 then install the core22 and install back the amazon-ssm-agent using candidate parameter

sudo snap install --channel=candidate amazon-ssm-agent --classic

However the concern is that we should not be doing this manually right? And there is no point in depending to an old and outdated core18 with known vulnerabilities, why not bake the amazon-ssm-agent to the recent core24 instead? there is no harm in doing so right? and core24 is stable anyways. Unless I am missing something the bigger picture?

jsf9k commented 1 month ago

However the concern is that we should not be doing this manually right?

We build our own AMIs (starting from a suitable base AMI offered by AWS) and as part of that we apply this Ansible role. This role installs the latest system package (*.deb or *.rpm) made available by AWS on AL2023, Debian, Fedora, and Kali. On Ubuntu the role installs the latest snap (from the candidate channel) since that's what is recommended in the documentation for Ubuntus of recent vintage.

And there is no point in depending to an old and outdated core18 with known vulnerabilities, why not bake the amazon-ssm-agent to the recent core24 instead? there is no harm in doing so right? and core24 is stable anyways. Unless I am missing something in a bigger picture?

I don't know what AMI you are using, but AWS doesn't actually create most of the available AMIs. They are created by third parties such as Debian, Fedora, etc. So AWS likely doesn't control what version of the SSM agent snap comes pre-installed on the AMI. You would have to take that up with whatever entity creates the AMI you're using.

AWS could likely make the latest version of the SSM agent snap depend on core24, though. I suggest creating a separate issue for that, since that change would get everyone closer to where we want to be.

nicanorflavier commented 1 month ago

However the concern is that we should not be doing this manually right?

We build our own AMIs (starting from a suitable base AMI offered by AWS) and as part of that we apply this Ansible role. This role installs the latest system package (*.deb or *.rpm) made available by AWS on AL2023, Debian, Fedora, and Kali. On Ubuntu the role installs the latest snap (from the candidate channel) since that's what is recommended in the documentation for Ubuntus of recent vintage.

And there is no point in depending to an old and outdated core18 with known vulnerabilities, why not bake the amazon-ssm-agent to the recent core24 instead? there is no harm in doing so right? and core24 is stable anyways. Unless I am missing something in a bigger picture?

I don't know what AMI you are using, but AWS doesn't actually create most of the available AMIs. They are created by third parties such as Debian, Fedora, etc. So AWS likely doesn't control what version of the SSM agent snap comes pre-installed on the AMI. You would have to take that up with whatever entity creates the AMI you're using.

AWS could likely make the latest version of the SSM agent snap depend on core24, though. I suggest creating a separate issue for that, since that change would get everyone closer to where we want to be.

This is the point of raising this case because AWS doesn’t control the core version in third-party AMIs, but they can update their amazon-ssm-agent snap to depend on newer versions like core24 I believe this is set in snapcraft. Then this would ensure that when third-party AMIs (like Ubuntu) are created, they pull in the latest SSM agent snap version with the appropriate core dependency right?

jsf9k commented 1 month ago

Yep, I think we're saying the same thing. AWS should upgrade to core24 for their latest release.

Aperocky commented 3 weeks ago

We have promoted a two new version (3.3.859.0, 3.3.987.0) to latest/stable, both of which and any further versions forthcoming will be depending on snap core22.

We'll evaluate moving to core24 in the future, however there are no current plans to migrate to core24.