CloudSnorkel / cdk-github-runners

CDK constructs for self-hosted GitHub Actions runners
https://constructs.dev/packages/@cloudsnorkel/cdk-github-runners/
Apache License 2.0
255 stars 37 forks source link

Missing amazon-cloudwatch-agent for Amazon Linux 2 and 2023 #530

Closed wash-amzn closed 1 month ago

wash-amzn commented 1 month ago

The below is with regard to version 0.12.4 and I am using Ec2RunnerProvider.

Ec2RunnerProvider's user-data script depends on amazon-cloudwatch-agent being installed, but RunnerImageComponent.requiredPackages() does not install it on Amazon Linux 2 or 2023 (it does install it on Ubuntu).

The result is that launched Amazon Linux instances fail, and do so in such a way that they don't shut themselves down. I don't have any more of the output, but the instance was sitting there in the heartbeat loop after this:

[   29.875652] cloud-init[1325]: + true
[   29.875805] cloud-init[1325]: + aws stepfunctions send-task-heartbeat --task-token <REDACTED>
[   29.876706] cloud-init[1325]: + /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:/tmp/log.conf
[   29.876861] cloud-init[1325]: /var/lib/cloud/instance/scripts/part-001: line 38: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl: No such file or directory
[   29.877005] cloud-init[1325]: + exit 2
[   29.877149] cloud-init[1325]: Apr 09 12:28:58 cloud-init[1325]: util.py[WARNING]: Failed running /var/lib/cloud/instance/scripts/part-001 [2]
[   29.877305] cloud-init[1325]: Apr 09 12:28:58 cloud-init[1325]: cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)
[   29.877456] cloud-init[1325]: Apr 09 12:28:58 cloud-init[1325]: util.py[WARNING]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python2.7/site-packages/cloudinit/config/cc_scripts_user.pyc'>) failed
[   29.881128] cloud-init[1325]: ci-info: no authorized ssh keys fingerprints found for user ec2-user.

The simple diff below resolved the issue for me (tested on Amazon Linux 2 and 2023):

diff --git a/node_modules/@cloudsnorkel/cdk-github-runners/lib/image-builders/components.js b/node_modules/@cloudsnorkel/cdk-github-runners/lib/image-builders/components.js
index f3534f3..e7cdbc4 100644
--- a/node_modules/@cloudsnorkel/cdk-github-runners/lib/image-builders/components.js
+++ b/node_modules/@cloudsnorkel/cdk-github-runners/lib/image-builders/components.js
@@ -72,13 +72,13 @@ class RunnerImageComponent {
                 else if (os.is(providers_1.Os.LINUX_AMAZON_2)) {
                     return [
                         'yum update -y',
-                        'yum install -y jq tar gzip bzip2 which binutils zip unzip sudo shadow-utils',
+                        'yum install -y jq tar gzip bzip2 which binutils zip unzip sudo shadow-utils amazon-cloudwatch-agent',
                     ];
                 }
                 else if (os.is(providers_1.Os.LINUX_AMAZON_2023)) {
                     return [
                         'dnf upgrade -y',
-                        'dnf install -y jq tar gzip bzip2 which binutils zip unzip sudo shadow-utils findutils',
+                        'dnf install -y jq tar gzip bzip2 which binutils zip unzip sudo shadow-utils findutils amazon-cloudwatch-agent',
                     ];
                 }
                 else if (os.is(providers_1.Os.WINDOWS)) {