Audit not failing in GitLab CI pipeline #345

Open MacPiston opened 1 month ago

Expected behavior: Audit should fail because of vulnerable dependencies detected in project. Output:

Failed security audit due to high vulnerabilities.
Vulnerable advisories are:
https://github.com/advisories/xxx
https://github.com/advisories/yyy
https://github.com/advisories/zzz
Exiting...

Acutal behavior: Audit passes despite detecting vulnerable dependencies in project. Output:

PNPM audit report summary:
{
  "vulnerabilities": {
    "info": 0,
    "low": 2,
    "moderate": 7,
    "high": 3,
    "critical": 0
  },
  "dependencies": 865,
  "devDependencies": 0,
  "optionalDependencies": 0,
  "totalDependencies": 865
}
Passed pnpm security audit.

Config:

{
  "$schema": "https://github.com/IBM/audit-ci/raw/main/docs/schema.json",
  "package-manager": "pnpm",
  "skip-dev": true,
  "high": true
}

Description: When using GitLab CI (self-hosted instance, gitlab-runner 17.3.1 + node:18-bullseye-slim) running audit-ci does not fail, even though summary correctly lists high vulnerabilities. Running exactly the same audit locally causes failure due to high vulnerabilities (expected behavior). It does not matter whether json or CLI config is used - audit-ci always fails to exit on detecting vulnerabilities when running on GitLab CI pipeline.

Project uses PNPM version 9.1.1 (although the same behavior has been observed on latest i.e. 9.12.1)

Same here in Gitea, why is this occuring..? Here is my step config, and the step log:

            - name: NodeJS package vulnerability scan
              run: pnpm dlx audit-ci@^7 --config ./.audit-ci.jsonc

Here is my audit-ci config:

{
    "$schema": "https://github.com/IBM/audit-ci/raw/main/docs/schema.json",
    "low": true,
    "package-manager": "pnpm",
    "report-type": "full",
    "allowlist": []
}

So far I haven't had the time to test it myself, but my guess is that it has something to do with reading from stdout - I'll try to check in a few days. That could explain why locally it works fine.

Any luck?

@quinnturner could you please look into this issue?

@xxfogs I don't have access to those runners, so I am not sure how to debug. If I had to guess, I'd say the stdout/stderr is the right place to look. I'd appreciate it if the community helped out in this one!

I seem to be reproducing this issue on my computer now too..

IBM / audit-ci

Audit not failing in GitLab CI pipeline #345