Problems with the domain_join.sh when using noexec on /tmp (CIS Benchmarks)

[michaelsmoody]

Good evening,

We're running into an issue (been there, done that, seen it before, so I had a reasonable idea of what to look for) where something is working fine on systems that are not hardened according to CIS Benchmarks (or, for that matter, DISA STIG, PCI, and a few others). This is due to the noexec mount option on /tmp

The domain_join.sh script that SSM creates/downloads places some files in /tmp, and then turns around and tries to execute from its placement in /tmp

Specifically: https://github.com/aws/amazon-ssm-agent/blob/215d334a3d8204f37883399a5b81882b47951650/agent/plugins/domainjoin/domainjoin_unix_script.go#L223

What this causes is that the installation doesn't complete, due to the attempted execution in /tmp. The proper course of action is to use an application-specific tmp folder, or to perform this action is some other way. For safety, exec of files in /tmp is considered a non-compliant method.

I would recommend that this be changed to execute from either /usr/local, the install location of SSM, or really, just about anywhere else.

This is definitely causing us a significant issue for seamless join, and while I understand that you can't compensate for every edge case and distribution, this is a change that I believe would be well received by just about anyone. It should be relatively trivial to move the execution from /tmp to compensate, and then simply to clean-up after ourselves.

We're trying to figure out what the best course of action is, but for the moment, we're getting a permission denied at that step.

/var/lib/amazon/ssm/i-lknjaosindn/document/orchestration/{OBFUSCATED}/domainJoin/aws_domainjoin.sh: line 199: ./aws/install: Permission denied\nexit status 1"

I can perhaps create a PR for moving it elsewhere, but I'm hoping that a more rapid respond might occur on your end to resolve.

Thanks, Michael

[smhmhmd]

Agreed, the aws install directory should be moved from /tmp, where would you like to move the tmp directory ?

[michaelsmoody]

My apologies for the late response, with the holiday in the USA.

Any other location would conceivably be acceptable. Some proposals are below:

/root #Less than ideal, but potentially appropriate since this is being executed as root /opt #probably reasonable, since it's for non-system manager installed software, also /opt/bin, or /opt/ssm-agent, etc /var/opt #might not exist on all systems, but used for similar purposes /usr/local #also an ideal situation, almost always exists, almost always available /usr/local/bin #probably the absolute best since this is ideally where it would belong

Thank you for your consideration in this matter!

[smhmhmd]

Thanks for getting back to us. Could you please point us to an AMI that you would like us to test with. I can move the tmp dir to /usr/local, (holds directories + less chance of noexec mount), so, the code would look like: curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "/usr/local/awscliv2.zip" cd /usr/local unzip awscliv2.zip ./aws/install -u 1>/dev/null if [ $? -ne 0 ]; then echo "***Failed: aws cli install" && exit 1; fi cd -

[michaelsmoody]

@smhmhmd Good evening!

Would you kindly test it on this AMI? ami-0a0a8eb6cbde7b7fb

(We do this on this via userdata to ensure SSM agent is installed):

#!/bin/bash -xe
exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1
sudo yum install -y https://s3.us-gov-west-1.amazonaws.com/amazon-ssm-us-gov-west-1/latest/linux_amd64/amazon-ssm-agent.rpm
sudo sed -i '/AllowUsers centos/d' ./etc/ssh/sshd_config

This is an image for CIS CentOS 7 Level 1 straight from Center from Internet Security. We're currently using a modified version the this script we inject as an SSM document to get it join instead.

Thanks in advance.

[smhmhmd]

@michaelsmoody We dont have access to paid AMIs for internal accounts, I will change the install dir to /usr/local and send for CR. If you can point me to a similar community AMI that would help as well.

[ferkhat-aws]

The fix was released as part of v3.0.655.0. Closing.

[harbinc]

I am having this exact same issue. Did anyone come up with a solution?

[harbinc]

Found the solution. Move and symlink SSM agent if deploying to hardened image where /var has noexec on mounted partition. – Apply before running Quick Setup. cd /var/lib/amazon sudo mv ssm /opt/ ln -s /opt/ssm ssm sudo systemctl restart amazon-ssm-agent

[smhmhmd]

/tmp is not used anymore

[harbinc]

/tmp is no longer used but the agent downloads and executes scripts from /var/lib/amazon/ssm which has noexec.