SSM document : AWS-RunPatchBaseline - meaning of exit code 196 ?

[copolycube]

Hello,

I'm not sure where would be the best place to ask a question I am currently facing with the AWS document run through the ssm agent "AWS-RunPatchBaseline", feel free to close the issue if this is not the proper place !

I am facing some issue with the SSM maintenance windows I configured and am progressively rolling out over a fleet of several hundreds of servers. I tried to segregate the issues as I encounter them and will hence focus on a single one I observed and I'm not able to find informations about it online so far.

All my maintenance windows are configured to "RebootIfNeeded", but I observe recurrently the following message :

[ERROR]: Patching Operation failed with exit code 196 but instance needs reboot

Would you know how I should interpret this ? Would anyone have a clue on the possible mitigations ? Those instance have been deployed manually (no cloudformation involved, no terraform involved), and where not rebooted externally.

Thanks in advance for your help !

Log extract from the end of the stdout logs pushed to an s3 bucket (sparing you the beginning with the multiple installation stages) :

[...]
01/16/2021 14:05:05 root https://forums.aws.amazon.com/: Saving inventory to local configuration directory
01/16/2021 14:05:05 root https://forums.aws.amazon.com/: Attempting full upload
01/16/2021 14:05:05 botocore.credentials https://forums.aws.amazon.com/: Found credentials in environment variables.
01/16/2021 14:05:06 root https://forums.aws.amazon.com/: Upload complete.
01/16/2021 14:05:06 root https://forums.aws.amazon.com/: Report upload successful.
01/16/2021 14:05:06 root https://forums.aws.amazon.com/: Inventory upload was successful
01/16/2021 14:05:06 root https://forums.aws.amazon.com/: Patching Operation failed with exit code 196 but instance needs reboot
/usr/bin/python3
/usr/bin/python2.7
/usr/bin/python2
/usr/bin/python
/usr/bin/apt-get
Reading package lists...
Building dependency tree...
Reading state information...
python3-apt is already the newest version (1.1.0~beta1ubuntu0.16.04.11).
The following packages were automatically installed and are no longer required:
linux-headers-4.4.0-190 linux-headers-4.4.0-190-generic
linux-headers-4.4.0-194 linux-headers-4.4.0-194-generic
linux-image-4.4.0-190-generic linux-image-4.4.0-194-generic
linux-modules-4.4.0-190-generic linux-modules-4.4.0-194-generic
linux-modules-extra-4.4.0-190-generic linux-modules-extra-4.4.0-194-generic
Use 'apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 23 not upgraded.
Using python binary: 'python3'
Using Python Version: Python 3.5.2
/usr/bin/curl
/usr/bin/wget
01/16/2021 14:05:58 root https://forums.aws.amazon.com/: Downloading payload from https://s3.dualstack.us-east-1.amazonaws.com/aws-ssm-us-east-1/patchbaselineoperations/linux/payloads/patch-baseline-operations-1.65.tar.gz

[copolycube]

This actually seems to be an issue on the SSM Document, not on the SSM agent. Closing the issue here since this project seems to be only related to the agent...

"Exit Code 196 means "Reboot With Dependency Failure", meaning the operation fails for the reason some package update patch manager tries to install on your instance fails because that package update has dependency on packages that are explicitly rejected (and blocked) by the baseline.

Here is a scenario where Exit Code 196 is observed: the patch baseline rejects some package "A", and the rejected patches action is selected as "Block", then if patch manager finds a package "B" that meets the criteria specified in your baseline, and updating "B" requires installing "A", the updates fails with Exit Code 196.

Edited by: ziyelinaws on Jan 21, 2021 8:20 AM "