search

snyk / nodejs-lockfile-parser

Generate a Snyk dependency tree from package-lock.json or yarn.lock file
Other
59 stars 28 forks source link

[🐛] yarn 2 lock file parsing issues #56

Closed regevbr closed 4 years ago

regevbr commented 4 years ago

Expected behaviour

Work well with yarn 2 lock files

Actual behaviour

yarn.lock parsing failed with an error: Unknown token: { line: 3, col: 2, type: 'INVALID', value: undefined } 3:2 in lockfile

Steps to reproduce

create a project with yarn 2 and a github actions workflows that uses the snyk/actions/node@master action

Found issues

You use @yarnpkg/lockfile to parse the lock file, but it doesn't support yarn 2 lock files as they contain

__metadata:
  version: 4

which the parser does not support.

Suggested solution

realize that the repo is using yarn 2 (the existence of .yarnrc.yml is the indicator) and then use a different parser (I couldn't find an equivalent in yarn 2) Removing the __metadata line manually still doesn't solve the issue, we need to use a different parser

Debug log

  snyk test { _: [ [Circular] ], debug: true } +0ms
  snyk Error running test { error: { InvalidUserInputError: yarn.lock parsing failed with an error: Unknown token: { line: 3, col: 2, type: 'INVALID', value: undefined } 3:2 in lockfile
    at YarnLockParser.parseLockFile (/snapshot/snyk/node_modules/snyk-nodejs-lockfile-parser/lib/parsers/yarn-lock-parse.ts:65:13)
    at Object.<anonymous> (/snapshot/snyk/node_modules/snyk-nodejs-lockfile-parser/lib/index.ts:67:45)
    at Generator.next (<anonymous>)
    at __awaiter (/snapshot/snyk/node_modules/tslib/tslib.js:113:75)
    at new Promise (<anonymous>)
    at Object.__awaiter (/snapshot/snyk/node_modules/tslib/tslib.js:109:16)
    at Object.buildDepTree (/snapshot/snyk/node_modules/snyk-nodejs-lockfile-parser/dist/index.js:20:20)
    at Object.parse (/snapshot/snyk/dist/lib/plugins/nodejs-plugin/npm-lock-parser.js:43:31)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:189:7) code: 422, name: 'InvalidUserInputError' } } +0ms
Error: 
Testing /app...

yarn.lock parsing failed with an error: Unknown token: { line: 3, col: 2, type: 'INVALID', value: undefined } 3:2 in lockfile
    at test (/snapshot/snyk/dist/cli/commands/test/index.js:173:23)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:189:7)
    at Function.Module.runMain (pkg/prelude/bootstrap.js:1317:13)
    at startup (bootstrap_node.js:240:16)
    at bootstrap_node.js:661:3
  snyk analytics { args: [ { debug: true, org: undefined, showVulnPaths: 'some' } ],
  command: 'bad-command',
  org: undefined,
  metadata: 
   { local: [ true, true ],
     'generating-node-dependency-tree': { lockFile: true, targetFile: 'yarn.lock' },
     'error-message': '\nTesting /app...\n\nyarn.lock parsing failed with an error: Unknown token: { line: 3, col: 2, type: \'INVALID\', value: undefined } 3:2 in lockfile',
     error: 'Error: \nTesting /app...\n\nyarn.lock parsing failed with an error: Unknown token: { line: 3, col: 2, type: \'INVALID\', value: undefined } 3:2 in lockfile\n    at test (/snapshot/snyk/dist/cli/commands/test/index.js:173:23)\n    at <anonymous>\n    at process._tickCallback (internal/process/next_tick.js:189:7)\n    at Function.Module.runMain (pkg/prelude/bootstrap.js:1317:13)\n    at startup (bootstrap_node.js:240:16)\n    at bootstrap_node.js:661:3',
     'error-code': 422,
     command: 'test' },
  version: '1.305.0',
  os: 'Linux 4.15',
  nodeVersion: 'v8.16.2',
  id: 'e8ed2f726c2573943ce493d9d7a6f5fa898b48d1',
  ci: false,
  durationMs: 44 } +0ms

  snyk Exit code: 2 +0ms
  snyk sending request to: https://snyk.io/api/v1/analytics/cli +0ms
  snyk request body size: 990 +0ms
  snyk gzipped request body size: 550 +0ms
  snyk not using proxy +1ms
regevbr commented 4 years ago

It seems that the "equivelnt" package for yarn 2 is @yarnpkg/parsers and the usage is:

const parsers = require('@yarnpkg/parsers');
parsers.parseSyml(file);

But it seems that the output is different than what you are expecting

dkontorovskyy commented 4 years ago

Was released in https://github.com/snyk/nodejs-lockfile-parser/pull/75#issuecomment-652907966

dkontorovskyy commented 4 years ago

Functionality is available in Snyk CLI since v1.361.0 🎉