With node_modules installed, my ash scan takes 6 min, but without node_modules installed, it takes 2 min. and 30 sec. I’d like it to always take 2 min 30 sec. Debug logs into thread. I'm running this locally on my beefy macBook Pro.
I think ash should take the same amount of time, whether with node_modules installed or not.
With node_modules:
pnpm security-scan
> @lh/monorepo@0.1.0 security-scan /Users/stickb/Code/dos/lighthouse
> pnpm dlx tsx scripts/src/security-scan
Packages: +6
++++++
Progress: resolved 28, reused 6, downloaded 0, added 6, done
latest: Pulling from ash
Digest: sha256:badd6ec03c62fa2964bb195a50b0c6bcc61c901d194b8d5ffad017e1fb6b1c92
Status: Image is up to date for 905418358903.dkr.ecr.us-east-1.amazonaws.com/ash:latest
905418358903.dkr.ecr.us-east-1.amazonaws.com/ash:latest
+ echo 'Resolved OCI_RUNNER to: docker'
Resolved OCI_RUNNER to: docker
+ '[' YES = NO ']'
+ RC=0
+ '[' NO = NO ']'
+ MOUNT_SOURCE_DIR='--mount type=bind,source=/Users/stickb/Code/dos/lighthouse,destination=/src'
+ MOUNT_OUTPUT_DIR=
+ OUTPUT_DIR_OPTION=
+ [[ NO = \Y\E\S ]]
+ echo 'Running ASH scan using built image...'
Running ASH scan using built image...
+ docker run --rm -e ACTUAL_SOURCE_DIR=/Users/stickb/Code/dos/lighthouse -e ASH_DEBUG=YES --mount type=bind,source=/Users/stickb/Code/dos/lighthouse,destination=/src 905418358903.dkr.ecr.us-east-1.amazonaws.com/ash:latest ash --source-dir /src
ASH version 1.3.3
*Cloning into '/tmp/ash-run-scan.nXNz'...
done.
Repository cloned successfully.
Imported ash-ignore-report.txt from /src/ash_output
Imported ash-scan-set-files-list.txt from /src/ash_output
ASH found 971 file(s) in the source directory...
Items to scan for in Dockerfile-cdk are: [ yaml yml json template ]
Items to scan for in Dockerfile-yaml are: [ yaml yml tf json dockerfile ]
Items to scan for in Dockerfile-git are: [ git ]
Running yaml-docker-execute.sh ...
Running cdk-docker-execute.sh ...
Items to scan for in Dockerfile-js are: [ js jsx ts tsx ]
Items to scan for in Dockerfile-py are: [ py pyc ipynb ]
Running git-docker-execute.sh ...
Running js-docker-execute.sh ...
waiting on Dockerfile-cdk to finish ...
Items to scan for in Dockerfile-grype are: [ js jsx ts tsx py java go cs sh war jar ]
Running py-docker-execute.sh ...
Running grype-docker-execute.sh ...
[2024-04-26 20:26:49] DEBUG: [js] pwd: '/tmp/ash-run-scan.nXNz' :: _ASH_SOURCE_DIR: /tmp/ash-run-scan.nXNz :: _ASH_RUN_DIR: /tmp/ash-run-scan.nXNz
[2024-04-26 20:26:49] DEBUG: [py] pwd: '/tmp/ash-run-scan.nXNz' :: _ASH_SOURCE_DIR: /tmp/ash-run-scan.nXNz :: _ASH_RUN_DIR: /tmp/ash-run-scan.nXNz
[2024-04-26 20:26:49] DEBUG: [git] pwd: '/tmp/ash-run-scan.nXNz' :: _ASH_SOURCE_DIR: /tmp/ash-run-scan.nXNz :: _ASH_RUN_DIR: /tmp/ash-run-scan.nXNz
[2024-04-26 20:26:49] DEBUG: [grype] pwd: '/tmp/ash-run-scan.nXNz' :: _ASH_SOURCE_DIR: /tmp/ash-run-scan.nXNz :: _ASH_RUN_DIR: /tmp/ash-run-scan.nXNz
[2024-04-26 20:26:49] DEBUG: [yaml] pwd: '/tmp/ash-run-scan.nXNz' :: _ASH_SOURCE_DIR: /tmp/ash-run-scan.nXNz :: _ASH_RUN_DIR: /tmp/ash-run-scan.nXNz
[2024-04-26 20:26:49] DEBUG: [cdk] pwd: '/tmp/ash-run-scan.nXNz' :: _ASH_SOURCE_DIR: /tmp/ash-run-scan.nXNz :: _ASH_RUN_DIR: /tmp/ash-run-scan.nXNz
[2024-04-26 20:26:49] DEBUG: Starting all scanners within the Grype scanner tool set
[2024-04-26 20:26:49] DEBUG: Starting Grype scan of /tmp/ash-run-scan.nXNz
[2024-04-26 20:26:49] DEBUG: Starting all scanners within the CDK scanner tool set
[2024-04-26 20:26:49] DEBUG: Found 0 CloudFormation files to scan:
Dockerfile Dockerfile-git returned 0
Dockerfile Dockerfile-py returned 0
Dockerfile Dockerfile-js returned 0
[2024-04-26 20:27:04] DEBUG: Finished all scanners within the CDK scanner tool set
Dockerfile Dockerfile-cdk returned 0
Dockerfile-cdk finished with return code 0
waiting on Dockerfile-yaml to finish ...
Dockerfile Dockerfile-yaml returned 0
Dockerfile-yaml finished with return code 0
waiting on Dockerfile-git to finish ...
Dockerfile-git finished with return code 0
waiting on Dockerfile-py to finish ...
Dockerfile-py finished with return code 0
waiting on Dockerfile-js to finish ...
Dockerfile-js finished with return code 0
waiting on Dockerfile-grype to finish ...
[2024-04-26 20:27:32] DEBUG: Finished Grype scan of /tmp/ash-run-scan.nXNz
[2024-04-26 20:27:32] DEBUG: Starting Grype scan of /src/ash_output/work
[2024-04-26 20:27:38] DEBUG: Finished Grype scan of /src/ash_output/work
[2024-04-26 20:27:38] DEBUG: Starting Syft scan of /tmp/ash-run-scan.nXNz
[2024-04-26 20:27:38] DEBUG: syft /tmp/ash-run-scan.nXNz --exclude="**/*-converted.py" --exclude="**/*_report_result.txt"
[2024-04-26 20:27:42] DEBUG: Finished Syft scan of /tmp/ash-run-scan.nXNz
[2024-04-26 20:27:42] DEBUG: Starting Syft scan of /src/ash_output/work
[2024-04-26 20:27:42] DEBUG: syft /src/ash_output/work --exclude="**/*-converted.py" --exclude="**/*_report_result.txt"
[2024-04-26 20:27:44] DEBUG: Finished Syft scan of /src/ash_output/work
[2024-04-26 20:27:44] DEBUG: Starting Semgrep scan of /tmp/ash-run-scan.nXNz
[2024-04-26 20:28:15] DEBUG: Finished Semgrep scan of /tmp/ash-run-scan.nXNz
[2024-04-26 20:28:15] DEBUG: Starting Semgrep scan of /src/ash_output/work
[2024-04-26 20:28:33] DEBUG: Finished Semgrep scan of /src/ash_output/work
[2024-04-26 20:28:33] DEBUG: Finished all scanners within the Grype scanner tool set
Dockerfile Dockerfile-grype returned 1
Dockerfile-grype finished with return code 1
Jobs return code report:
Dockerfile-cdk : 0
Dockerfile-yaml : 0
Dockerfile-git : 0
Dockerfile-py : 0
Dockerfile-js : 0
Dockerfile-grype : 1
Your final report can be found here: /src/ash_output/aggregated_results.txt
ASH execution completed in 329 seconds.
Highest return code is 1
+ RC=1
+ [[ YES = \Y\E\S ]]
+ set +x
Error: Command failed: ASH_IMAGE_NAME=905418358903.dkr.ecr.us-east-1.amazonaws.com/ash:latest ./ash.sh --no-build --debug --oci-runner docker
at __node_internal_genericNodeError (node:internal/errors:932:15)
at checkExecSyncError (node:child_process:890:11)
at execSync (node:child_process:962:15)
at <anonymous> (/Users/stickb/Code/dos/lighthouse/scripts/src/security-scan.ts:24:3)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
status: 1,
signal: null,
output: [ null, null, null ],
pid: 48082,
stdout: null,
stderr: null
}
node:internal/process/esm_loader:34
internalBinding('errors').triggerUncaughtException(
^
Error: Command failed: ASH_IMAGE_NAME=905418358903.dkr.ecr.us-east-1.amazonaws.com/ash:latest ./ash.sh --no-build --debug --oci-runner docker
at __node_internal_genericNodeError (node:internal/errors:932:15)
at checkExecSyncError (node:child_process:890:11)
at execSync (node:child_process:962:15)
at <anonymous> (/Users/stickb/Code/dos/lighthouse/scripts/src/security-scan.ts:24:3)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
status: 1,
signal: null,
output: [ null, null, null ],
pid: 48082,
stdout: null,
stderr: null
}
Node.js v20.11.1
ELIFECYCLE Command failed with exit code 1.
Without node_modules:
pnpm security-scan
> @lh/monorepo@0.1.0 security-scan /Users/stickb/Code/dos/lighthouse-no-deps
> pnpm dlx tsx scripts/src/security-scan
Packages: +6
++++++
Progress: resolved 28, reused 6, downloaded 0, added 6, done
latest: Pulling from ash
Digest: sha256:badd6ec03c62fa2964bb195a50b0c6bcc61c901d194b8d5ffad017e1fb6b1c92
Status: Image is up to date for 905418358903.dkr.ecr.us-east-1.amazonaws.com/ash:latest
905418358903.dkr.ecr.us-east-1.amazonaws.com/ash:latest
+ echo 'Resolved OCI_RUNNER to: docker'
Resolved OCI_RUNNER to: docker
+ '[' YES = NO ']'
+ RC=0
+ '[' NO = NO ']'
+ MOUNT_SOURCE_DIR='--mount type=bind,source=/Users/stickb/Code/dos/lighthouse-no-deps,destination=/src'
+ MOUNT_OUTPUT_DIR=
+ OUTPUT_DIR_OPTION=
+ [[ NO = \Y\E\S ]]
+ echo 'Running ASH scan using built image...'
Running ASH scan using built image...
+ docker run --rm -e ACTUAL_SOURCE_DIR=/Users/stickb/Code/dos/lighthouse-no-deps -e ASH_DEBUG=YES --mount type=bind,source=/Users/stickb/Code/dos/lighthouse-no-deps,destination=/src 905418358903.dkr.ecr.us-east-1.amazonaws.com/ash:latest ash --source-dir /src
ASH version 1.3.3
Cloning into '/tmp/ash-run-scan.B811'...
done.
Repository cloned successfully.
ASH found 486 file(s) in the source directory...
Items to scan for in Dockerfile-cdk are: [ yaml yml json template ]
Items to scan for in Dockerfile-yaml are: [ yaml yml tf json dockerfile ]
Items to scan for in Dockerfile-py are: [ py pyc ipynb ]
Items to scan for in Dockerfile-git are: [ git ]
Running cdk-docker-execute.sh ...
Running yaml-docker-execute.sh ...
Items to scan for in Dockerfile-js are: [ js jsx ts tsx ]
Running git-docker-execute.sh ...
waiting on Dockerfile-cdk to finish ...
Running py-docker-execute.sh ...
Items to scan for in Dockerfile-grype are: [ js jsx ts tsx py java go cs sh war jar ]
Running grype-docker-execute.sh ...
Running js-docker-execute.sh ...
[2024-04-26 20:30:02] DEBUG: [yaml] pwd: '/tmp/ash-run-scan.B811' :: _ASH_SOURCE_DIR: /tmp/ash-run-scan.B811 :: _ASH_RUN_DIR: /tmp/ash-run-scan.B811
[2024-04-26 20:30:02] DEBUG: [js] pwd: '/tmp/ash-run-scan.B811' :: _ASH_SOURCE_DIR: /tmp/ash-run-scan.B811 :: _ASH_RUN_DIR: /tmp/ash-run-scan.B811
[2024-04-26 20:30:02] DEBUG: [py] pwd: '/tmp/ash-run-scan.B811' :: _ASH_SOURCE_DIR: /tmp/ash-run-scan.B811 :: _ASH_RUN_DIR: /tmp/ash-run-scan.B811
[2024-04-26 20:30:02] DEBUG: [git] pwd: '/tmp/ash-run-scan.B811' :: _ASH_SOURCE_DIR: /tmp/ash-run-scan.B811 :: _ASH_RUN_DIR: /tmp/ash-run-scan.B811
[2024-04-26 20:30:02] DEBUG: [cdk] pwd: '/tmp/ash-run-scan.B811' :: _ASH_SOURCE_DIR: /tmp/ash-run-scan.B811 :: _ASH_RUN_DIR: /tmp/ash-run-scan.B811
[2024-04-26 20:30:02] DEBUG: [grype] pwd: '/tmp/ash-run-scan.B811' :: _ASH_SOURCE_DIR: /tmp/ash-run-scan.B811 :: _ASH_RUN_DIR: /tmp/ash-run-scan.B811
[2024-04-26 20:30:02] DEBUG: Starting all scanners within the Grype scanner tool set
[2024-04-26 20:30:03] DEBUG: Starting Grype scan of /tmp/ash-run-scan.B811
[2024-04-26 20:30:03] DEBUG: Starting all scanners within the CDK scanner tool set
[2024-04-26 20:30:03] DEBUG: Found 0 CloudFormation files to scan:
Dockerfile Dockerfile-git returned 0
Dockerfile Dockerfile-py returned 0
Dockerfile Dockerfile-js returned 0
[2024-04-26 20:30:11] DEBUG: Finished all scanners within the CDK scanner tool set
Dockerfile Dockerfile-cdk returned 0
Dockerfile-cdk finished with return code 0
waiting on Dockerfile-yaml to finish ...
Dockerfile Dockerfile-yaml returned 0
Dockerfile-yaml finished with return code 0
waiting on Dockerfile-git to finish ...
Dockerfile-git finished with return code 0
waiting on Dockerfile-py to finish ...
Dockerfile-py finished with return code 0
waiting on Dockerfile-js to finish ...
Dockerfile-js finished with return code 0
waiting on Dockerfile-grype to finish ...
[2024-04-26 20:30:41] DEBUG: Finished Grype scan of /tmp/ash-run-scan.B811
[2024-04-26 20:30:41] DEBUG: Starting Grype scan of /src/ash_output/work
[2024-04-26 20:30:46] DEBUG: Finished Grype scan of /src/ash_output/work
[2024-04-26 20:30:46] DEBUG: Starting Syft scan of /tmp/ash-run-scan.B811
[2024-04-26 20:30:46] DEBUG: syft /tmp/ash-run-scan.B811 --exclude="**/*-converted.py" --exclude="**/*_report_result.txt"
[2024-04-26 20:30:50] DEBUG: Finished Syft scan of /tmp/ash-run-scan.B811
[2024-04-26 20:30:50] DEBUG: Starting Syft scan of /src/ash_output/work
[2024-04-26 20:30:50] DEBUG: syft /src/ash_output/work --exclude="**/*-converted.py" --exclude="**/*_report_result.txt"
[2024-04-26 20:30:52] DEBUG: Finished Syft scan of /src/ash_output/work
[2024-04-26 20:30:52] DEBUG: Starting Semgrep scan of /tmp/ash-run-scan.B811
[2024-04-26 20:31:31] DEBUG: Finished Semgrep scan of /tmp/ash-run-scan.B811
[2024-04-26 20:31:31] DEBUG: Starting Semgrep scan of /src/ash_output/work
[2024-04-26 20:31:53] DEBUG: Finished Semgrep scan of /src/ash_output/work
[2024-04-26 20:31:53] DEBUG: Finished all scanners within the Grype scanner tool set
Dockerfile Dockerfile-grype returned 1
Dockerfile-grype finished with return code 1
Jobs return code report:
Dockerfile-cdk : 0
Dockerfile-yaml : 0
Dockerfile-git : 0
Dockerfile-py : 0
Dockerfile-js : 0
Dockerfile-grype : 1
Your final report can be found here: /src/ash_output/aggregated_results.txt
ASH execution completed in 119 seconds.
Highest return code is 1
+ RC=1
+ [[ YES = \Y\E\S ]]
+ set +x
Error: Command failed: ASH_IMAGE_NAME=905418358903.dkr.ecr.us-east-1.amazonaws.com/ash:latest ./ash.sh --no-build --debug --oci-runner docker
at __node_internal_genericNodeError (node:internal/errors:932:15)
at checkExecSyncError (node:child_process:890:11)
at execSync (node:child_process:962:15)
at <anonymous> (/Users/stickb/Code/dos/lighthouse-no-deps/scripts/src/security-scan.ts:24:3)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
status: 1,
signal: null,
output: [ null, null, null ],
pid: 52556,
stdout: null,
stderr: null
}
node:internal/process/esm_loader:34
internalBinding('errors').triggerUncaughtException(
^
Error: Command failed: ASH_IMAGE_NAME=905418358903.dkr.ecr.us-east-1.amazonaws.com/ash:latest ./ash.sh --no-build --debug --oci-runner docker
at __node_internal_genericNodeError (node:internal/errors:932:15)
at checkExecSyncError (node:child_process:890:11)
at execSync (node:child_process:962:15)
at <anonymous> (/Users/stickb/Code/dos/lighthouse-no-deps/scripts/src/security-scan.ts:24:3)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
status: 1,
signal: null,
output: [ null, null, null ],
pid: 52556,
stdout: null,
stderr: null
}
Node.js v20.11.1
ELIFECYCLE Command failed with exit code 1.
WARN Local package.json exists, but node_modules missing, did you mean to install?
Hey ASH team, thanks for the great work 1.3.2!
With node_modules installed, my ash scan takes 6 min, but without node_modules installed, it takes 2 min. and 30 sec. I’d like it to always take 2 min 30 sec. Debug logs into thread. I'm running this locally on my beefy macBook Pro.
I think ash should take the same amount of time, whether with node_modules installed or not.
With node_modules:
Without node_modules: