plugin doesn't work on slaves, only on build-in(master)

[spu-xb01]

Jenkins and plugins versions report

Jenkins: 2.440.1 OS: Linux - 5.10.205-195.807.amzn2.aarch64 Java: 17.0.10 - Eclipse Adoptium (OpenJDK 64-Bit Server VM) amazon-ecr:1.114.vfd22430621f5 amazon-inspector-image-scanner:297.vb_d6333ef3680

What Operating System are you using (both controller, and any agents involved in the problem)?

NAME="Amazon Linux" VERSION="2" ID="amzn" ID_LIKE="centos rhel fedora" VERSION_ID="2" PRETTY_NAME="Amazon Linux 2" ANSI_COLOR="0;33" CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2" HOME_URL="https://amazonlinux.com/" SUPPORT_END="2025-06-30" Amazon Linux release 2 (Karoo)

Reproduction steps

1.launching a pipeline on a slave

2. all actions take place on the master

Replayed [#106](http://jenkins.com/job/cdmnext-amazon-inspector-scan/106/)
[Pipeline] Start of Pipeline
[Pipeline] node
Running on [STAGE-ARM64](http://jenkins.com/computer/STAGE%2DARM64/) in /home/ec2-user/workspace/cdmnext-amazon-inspector-scan
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Declarative: Checkout SCM) ([hide](http://jenkins.com/job/cdmnext-amazon-inspector-scan/107/console#))
[Pipeline] checkout
The recommended git tool is: git
Fetching changes from the remote Git repository
Checking out Revision a2b8af0aa3c97c26eead02a1e8799b172e3f938a (refs/remotes/origin/master)

Expected Results

plugin should run on the slave node and download Amazon Inspector SBOM Generator to the slave, not to the master

Actual Results

pipeline {
    agent {
        label 'ARM64'
    }
    environment {
        AWS_REGION = 'us-east-1'
        ECR_REPO = 'arm64-base'
        ID = 'xxxxxxxxxx'
        URL_REGISTRY = "xxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com"
    }

    stages {
        stage ('amazon-inspector-image-scanner') {
            steps {
                script {
                    step ([
                        $class: 'com.amazon.inspector.jenkins.amazoninspectorbuildstep.AmazonInspectorBuilder',
                        sbomgenSource: 'linuxArm64',
                        archivePath: '${URL_REGISTRY}/arm64-base:base-image',
                        awsRegion: 'us-east-1',
                        iamRole: 'arn:aws:iam::${ID}:role/dev-ecr-role',
                        credentialId: '',
                        isThresholdEnabled: 'false',
                        countCritical: 0, 
                        countHigh: 0, 
                        countLow: 10, 
                        countMedium: 5,
                    ])
                }
            }
        }
    }
}    

when specifying another slave, the plugin downloads all files to the master; when launched manually, the plugin searches for Amazon Inspector SBOM Generator binary files on the master, not on the slave

Anything else?

No response

Are you interested in contributing a fix?

No response

[waltwilo]

Hi, could you please provide the logs for the run where this happened?

[spu-xb01]

@waltwilo I don’t know what other log can be added, but the pipeline is started on the slave, but the folder where the generator is downloaded is the master’s folder. If I manually specify the path, the plugin looks for it on the master, not on the slave

Started by user [admin](http://jenkins.com/user/admin)
Obtained jenkins/jobs/scan/Jenkinsfile-auth from git git@bitbucket.org:repo/deploy.git
[Pipeline] Start of Pipeline
[Pipeline] node
Running on [ARM64](http://jenkins.com/computer/ARM64/) in /home/ec2-user/workspace/cdmnext-amazon-inspector-scan
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Declarative: Checkout SCM)
[Pipeline] checkout
The recommended git tool is: git
Fetching changes from the remote Git repository
 > git --version # timeout=10
 > git --version # 'git version 2.40.1'
using GIT_SSH to set credentials cdmnext
Checking out Revision 31d3c839d12
Commit message: "Fix"
[Pipeline] }
[Pipeline] // stage
[Pipeline] withEnv
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Docker image pull)
[Pipeline] sh
+ aws ecr get-login-password --region us-east-1
+ docker login --username AWS --password-stdin ****************.dkr.ecr.us-east-1.amazonaws.com
WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (amazon-inspector-image-scanner)
[Pipeline] script
[Pipeline] {
[Pipeline] step
Automatic SBOMGen Sourcing selected, downloading now...
Credential ID is null, this is not normal, please check your config. Continuing without docker credentials.
No credential provided, running without.
Making downloaded SBOMGen executable...
Running command...
[/tmp/sbomgen2002781832029439579/inspector_sbomgen/inspector-sbomgen-1.2.0/linux/arm64/inspector-sbomgen, container, --image, 419280616994.dkr.ecr.us-east-1.amazonaws.com/cdmnext-arm64-base:cdmnext-base-node-image]
Plugin execution ran into an error and is being aborted!
Exception:com.amazon.inspector.jenkins.amazoninspectorbuildstep.exception.MalformedScanOutputException: Sbom scanning output formatted incorrectly.
Sbom Content:
time="2024-06-05T07:14:46Z" level=info msg="Amazon Inspector SBOM Generator v1.2.0 - linux arm64 - Copyright 2024 Amazon.com, Inc. or its affiliates. All Rights Reserved"
time="2024-06-05T07:14:46Z" level=info msg="[/tmp/sbomgen2002781832029439579/inspector_sbomgen/inspector-sbomgen-1.2.0/linux/arm64/inspector-sbomgen container --image 419280616994.dkr.ecr.us-east-1.amazonaws.com/cdmnext-arm64-base:cdmnext-base-node-image]"
time="2024-06-05T07:14:46Z" level=info msg="writing log file to: /var/jenkins_home/.inspector-sbomgen/logs/inspector-sbomgen-log_2024-06-05_07-14-46.txt"
time="2024-06-05 07:14:46" level=info msg="initializing target artifact" file="coreV1.go:34:"
time="2024-06-05 07:14:46" level=info msg="created temporary staging directory: /var/jenkins_home/.inspector-sbomgen/artifact-cache3437680771" file="stagingdir.go:60:"
time="2024-06-05 07:14:46" level=info msg="checking if image is a tarball" file="imageInit.go:28:"
time="2024-06-05 07:14:46" level=info msg="checking if image exists in the local Docker daemon" file="imageInit.go:37:"
time="2024-06-05 07:14:46" level=info msg="checking if image can be downloaded from a remote registry" file="imageInit.go:46:"

[waltwilo]

It looks like its at least able to run Inspector Sbomgen which should mean its able to find the binary if its running on the agent.

Would you mind posting the logs located at the location below? If they seem too sensitive to post on github, feel free to open a ticket to aws support and it will make its way to the inspector-seceng team.

/var/jenkins_home/.inspector-sbomgen/logs/inspector-sbomgen-log_2024-06-05_07-14-46.txt

[spu-xb01]

@waltwilo

time="2024-06-05 07:24:54" level=info msg="initializing target artifact" file="coreV1.go:34:"
time="2024-06-05 07:24:54" level=info msg="created temporary staging directory: /var/jenkins_home/.inspector-sbomgen/artifact-cache662253273" file="stagingdir.go:60:"
time="2024-06-05 07:24:54" level=info msg="checking if image is a tarball" file="imageInit.go:28:"
time="2024-06-05 07:24:54" level=info msg="checking if image exists in the local Docker daemon" file="imageInit.go:37:"
time="2024-06-05 07:24:54" level=info msg="checking if image can be downloaded from a remote registry" file="imageInit.go:46:"
time="2024-06-05 07:24:54" level=info msg="downloading remote container image: *************.dkr.ecr.us-east-1.amazonaws.com/arm64-base:base-node-image" file="imageInit.go:153:"

I checked several log files and they are the same

[waltwilo]

Hi, thanks for sending that. Based on the logs it seems like the binary is stuck downloading the image. The plugin should throw an error that's visible to you if something isn't working. Could you try running the plugin on a smaller image, like alpine:latest?

[waltwilo]

Closing this issue for now, feel free to re-open if the problem persists.

[ophintor]

I have exactly the same problem as @spu-xb01.

The plugin is run from an agent but it looks for the binary in the master controller. Initially I was getting the error of 'not found'. When I realised the problem was that it was looking for the binary in the master, I copied it there and it finds it, but obviously the master does not have the permissions to access the repo or anything so the plugin still fails with a 401.

In any case we don't need or want to run this in the master. If the plugin is executed on an agent then the binary should be found and executed in the agent.

[ophintor]

@waltwilo could we please re-open this issue?

[waltwilo]

I'm taking another look at this. I believe I know what the issue is now and I'll link the PR once I have it merged.

[waltwilo]

I have a potential fix ready to go and will be merging it later today.

[waltwilo]

PR: https://github.com/jenkinsci/amazon-inspector-image-scanner-plugin/pull/76

[waltwilo]

The above PR has been merged, let me know if this fixes your issues.