Open dwaynebailey opened 5 years ago
Can #6632 be rolled into this feature?
There's work in progress to integrate https://www.npmjs.com/package/npm-audit-resolver (which extends npm audit to do exactly this sort of thing) into npm, but it only works for npm-generated lockfiles. It would be awesome to have that same functionality for yarn lockfiles.
For anyone who needs to work around this now, we're using this:
set +e
yarn audit
result=$?
set -e
if [ "$result" != 0 ]; then
if [ -f yarn-audit-known-issues ]; then
set +e
yarn audit --json | grep auditAdvisory > yarn-audit-issues
set -e
if diff -q yarn-audit-known-issues yarn-audit-issues > /dev/null 2>&1; then
echo
echo Ignorning known vulnerabilities
exit 0
fi
fi
echo
echo Security vulnerabilities were found that were not ignored
echo
echo Check to see if these vulnerabilities apply to production
echo and/or if they have fixes available. If they do not have
echo fixes and they do not apply to production, you may ignore them
echo
echo To ignore these vulnerabilities, run:
echo
echo "yarn audit --json | grep auditAdvisory > yarn-audit-known-issues"
echo
echo and commit the yarn-audit-known-issues file
exit "$result"
fi
How is this not a thing yet?!
@AaronHarris Thank you very much for your snippet! I've modified your version to avoid run audit twice and create temporary file:
#!/bin/bash
# workaround for missing feature
# https://github.com/yarnpkg/yarn/issues/6669
set -u
set +e
output=$(yarn audit --json)
result=$?
set -e
if [ $result -eq 0 ]; then
# everything is fine
exit 0
fi
if [ -f yarn-audit-known-issues ] && echo "$output" | grep auditAdvisory | diff -q yarn-audit-known-issues - > /dev/null 2>&1; then
echo
echo Ignorning known vulnerabilities
exit 0
fi
echo
echo Security vulnerabilities were found that were not ignored
echo
echo Check to see if these vulnerabilities apply to production
echo and/or if they have fixes available. If they do not have
echo fixes and they do not apply to production, you may ignore them
echo
echo To ignore these vulnerabilities, run:
echo
echo "yarn audit --json | grep auditAdvisory > yarn-audit-known-issues"
echo
echo and commit the yarn-audit-known-issues file
echo
echo "$output" | grep auditAdvisory | python -mjson.tool
exit "$result"
@Bessonov That's great, thanks. We've put this into a CircleCI orb, for any using CircleCI: https://github.com/substantial/circleci-orbs/blob/master/yarn.yml
It includes a fail-safe for when yarn/npm servers are down so that builds don't fail in that case. If anyone wants to PR Bessonov's changes, that'd be great.
FYI there's a PR open for this https://github.com/yarnpkg/yarn/pull/8223
To fix this issue i have opened PR #8368, the PR fails but it seems unrelated to my changes (as other prs seem to fail on the same). If somebody can give feedback/review so i can change anything if needed that would be great.
This would be great in scenarios where the CVE is considered low-risk/irrelevant for a project, and no upstream fixes are yet available. For example: https://github.com/rails/webpacker/issues/3017 (postcss regex DoS vuln, dependency of @rails/webpacker)
@athix I don't know if there is an equivalent with yarn (I moved off of it), but here is what I use to do exactly that with npm: https://www.npmjs.com/package/better-npm-audit
Probably not super helpful, but thought I would share anyway. Just in case it can be helpful for someone.
+1 to @jipiboily suggestion. We moved to it last week and much happier. Yarn should just make that the built-in default.
audit-ci and npm-audit-resolver provides the same features as better-npm-audit with yarn support 😉
In Yarn 2, the analogous command seems to be yarn npm audit
(docs). It was added by https://github.com/yarnpkg/berry/pull/1892, and released in 2.4.0. It can omit devDependencies
, a common source of audit frustration.
yarn npm audit --environment production
Yarn 2 also has a plugin system which could be used to implement an alternative audit command.
@AaronHarris Thank you very much for your snippet! I've modified your version to avoid run audit twice and create temporary file:
#!/bin/bash # workaround for missing feature # https://github.com/yarnpkg/yarn/issues/6669 set -u set +e output=$(yarn audit --json) result=$? set -e if [ $result -eq 0 ]; then # everything is fine exit 0 fi if [ -f yarn-audit-known-issues ] && echo "$output" | grep auditAdvisory | diff -q yarn-audit-known-issues - > /dev/null 2>&1; then echo echo Ignorning known vulnerabilities exit 0 fi echo echo Security vulnerabilities were found that were not ignored echo echo Check to see if these vulnerabilities apply to production echo and/or if they have fixes available. If they do not have echo fixes and they do not apply to production, you may ignore them echo echo To ignore these vulnerabilities, run: echo echo "yarn audit --json | grep auditAdvisory > yarn-audit-known-issues" echo echo and commit the yarn-audit-known-issues file echo echo "$output" | grep auditAdvisory | python -mjson.tool exit "$result"
How did you set yarn-audit-known-issues ?
I followed the instructions in the echo message:
echo "yarn audit --json | grep auditAdvisory > yarn-audit-known-issues"
It looks like someone mentioned better-npm-audit, but there is also a similar package for yarn: improved-yarn-audit.
Use audit-ci
for running audits, it's developed by IBM: https://github.com/IBM/audit-ci
Use yarn-audit-fix
to fix auto-fixable vulnerabilities: https://github.com/antongolub/yarn-audit-fix, this is an awesome tool.
If you want to know more, take a look at https://debricked.com/blog/fixing-security-vulnerabilities-yarn/. It's a marketing blog-post but still useful.
We run audit before running install so adding new tools for doing the audit is a bit cyclical. Can be solved with a globally installed package or npx
but it's not ideal. This feature should really be baked in. I feel like there's bogus CVEs every other day now.
We run audit before running install so adding new tools for doing the audit is a bit cyclical. Can be solved with a globally installed package or
npx
but it's not ideal. This feature should really be baked in. I feel like there's bogus CVEs every other day now.
Can you provide examples? Do you mean CVEs that are low quality? Low severity? Not actually vulnerabilities? I'm asking because I want to avoid this problem if possible with the UVI project.
We run audit before running install so adding new tools for doing the audit is a bit cyclical. Can be solved with a globally installed package or
npx
but it's not ideal. This feature should really be baked in. I feel like there's bogus CVEs every other day now.Can you provide examples? Do you mean CVEs that are low quality? Low severity? Not actually vulnerabilities? I'm asking because I want to avoid this problem if possible with the UVI project.
Here is an example from today, this is a CVE that isn't actually a vulnerability but results in a critical warning in yarn audit
. It's pending revocation.
https://github.com/lodash/lodash/issues/5261
https://nvd.nist.gov/vuln/detail/CVE-2021-41720
We run audit before running install so adding new tools for doing the audit is a bit cyclical. Can be solved with a globally installed package or
npx
but it's not ideal. This feature should really be baked in. I feel like there's bogus CVEs every other day now.Can you provide examples? Do you mean CVEs that are low quality? Low severity? Not actually vulnerabilities? I'm asking because I want to avoid this problem if possible with the UVI project.
Here is an example from today, this is a CVE that isn't actually a vulnerability but results in a critical warning in
yarn audit
. It's pending revocation. lodash/lodash#5261 https://nvd.nist.gov/vuln/detail/CVE-2021-41720
Ok, interesting, I would point out it's in the disputed state so the process is proceeding, are there many more examples, e.g. how problematic is this? (1? 10? 100?)
Yeah we hit the lodash one too today and couple of days ago there was (and has now popped back apparently) Prototype pollution in objection.js, which is also marked as critical even though it's based on a prvate, internal function of the library. No one has demonstrated that this can actually be used to exploit the app through the lib's public API (see the maintainer's comments in the huntr thread).
At least in our setup high & critical vulns literally stop all builds, so it's pretty bad. I haven't bothered to set up the ignore-thingy thus far because these haven't been so common (like one or two times a year maybe), but now there's been a couple in the past few days so I'm going to bite the bullet before it gets worse.
Edit: But in addition to completely bogus vulns, there are also those that you just know don't affect you in any way. Like a dependency of Webpack has as a regex ddos vulnerability? What are the attackers going to do, inject malicious regex to your source code so your build halts and fans spin up? oh no
I think it's important that the ignoring functionality is also able to ignore a vulnerability given that the library is only a dependency of certain libraries. Like "ignore xyz if it's coming from one of webpack's dependencies".
Here is our slightly modified version which makes a cleaner git diff when you update your ignores
#!/usr/bin/env bash
set -eu -o pipefail
echo "--- Auditing yarn dependencies"
# https://github.com/yarnpkg/yarn/issues/6669#issuecomment-463684767
set +e
yarn audit
result=$?
if [ "$result" != 0 ]; then
yarn audit --high --json | grep auditAdvisory > yarn-audit-issues
ruby lib/yarn_audit/known_issue_processor.rb yarn-audit-issues yarn-audit-issues.csv
DIFF=$(diff yarn-audit-issues.csv yarn-audit-known-issues.csv)
if [ "${DIFF}" != "" ]; then
echo "Changes found on yarn-audit-issues"
echo "${DIFF}"
else
echo "No new issues that are not known"
exit 0
fi
echo Security vulnerabilities were found that were not ignored
echo
echo Check to see if these vulnerabilities apply to production
echo and/or if they have fixes available. If they do not have
echo fixes and they do not apply to production, you may ignore them
echo
echo To ignore these vulnerabilities, run locally:
echo
echo "bin/ci/yarn-audit; cp yarn-audit-issues.csv yarn-audit-known-issues.csv"
echo
echo and commit the yarn-audit-known-issues.csv
fi
exit "$result"
# frozen_string_literal: true
require "json"
require "csv"
module YarnAudit
class KnownIssueProcessor
def self.run(input_file_path, output_file_path)
csv_str = File.read(input_file_path).split("\n").map do |line|
json = JSON.parse(line)
advisory = json.dig("data", "advisory")
[advisory["module_name"], advisory["vulnerable_versions"], advisory["recommendation"], advisory["url"]].to_csv
end.uniq.sort.join
File.open(output_file_path, "w") do |f|
f.write(csv_str)
end
end
end
end
if __FILE__ == $0
YarnAudit::KnownIssueProcessor.run(ARGV[0], ARGV[1])
end
I just stumbled across https://github.com/yarnpkg/berry/blob/4d0eb0928543c37f8aedfac4a4a30c2731ac0772/packages/plugin-npm-cli/sources/commands/npm/audit.ts#L56
which seems to respond to the initial request. Could be adapted via a CI script to read from a file I guess
I changed @Bessonov script to use jq
instead of python:
#!/bin/bash
# workaround for missing feature
# https://github.com/yarnpkg/yarn/issues/6669
set -u
set +e
output=$(yarn audit --json --groups dependencies)
result=$?
set -e
if [ $result -eq 0 ]; then
# everything is fine
exit 0
fi
if [ -f yarn-audit-known-issues ] && echo "$output" | grep auditAdvisory | diff -q yarn-audit-known-issues - > /dev/null 2>&1; then
echo
echo Ignorning known vulnerabilities
jq --slurp '.[].data.advisory | {findings, title, severity, patched_versions}' yarn-audit-known-issues
exit 0
fi
echo
echo Security vulnerabilities were found that were not ignored
echo
echo Check to see if these vulnerabilities apply to production
echo and/or if they have fixes available. If they do not have
echo fixes and they do not apply to production, you may ignore them
echo
echo To ignore these vulnerabilities, run:
echo
echo "yarn audit --json --groups dependencies | grep auditAdvisory > yarn-audit-known-issues"
echo
echo and commit the yarn-audit-known-issues file
echo
echo "$output" | grep auditAdvisory | jq --slurp '.[].data.advisory | {findings, title, severity, patched_versions}'
exit "$result"
Do you want to request a feature or report a bug?
feature
What is the current behavior?
yarn audit
will report all issues and there is no way to suppress an issue that does not impact your code base.If the current behavior is a bug, please provide the steps to reproduce.
What is the expected behavior? Users can maintain a file like
.nsprc
or.snyk
to suppress various issues. This allows users to investigate and suppress issues that don't impact the code base or for which there is no current solution. This would also allow CI to be used to highlight new issues reported against the code base.Ideally the report should be able to be: 1) Suppressed 2) Suppressed for a period
Please mention your node.js, yarn and operating system version.