phylum-dev / cli

Command line interface for the Phylum API
https://phylum.io
GNU General Public License v3.0
101 stars 10 forks source link

Parse manifest files with non-standard names #1337

Closed maxrake closed 7 months ago

maxrake commented 7 months ago

This change allows for parsing manifest files with names that don't match the expected standards. For example, pip manifest files can be named anything and they are therefore hard to identify. Pip lockfiles can also be named anything but they will at least get picked up by the fallback approach of attempting to parse with all known parsers (assuming a "true" lockfile with no loose requirement entries).

A new custom error (UnknownManifestFormat) and exit code (21) were created to be used when a dependency file is provided without a file type and does not parse as a known lockfile, making it a likely manifest file.

A change was made to report errors with the pip version check that happens in the pip lockfile generator.

Additionally, a change was made to ensure ManifestWithoutGeneration errors use the display path instead of the full path.

Prerequisite checks were also added for lockfile generation to account for known manifest file names. Previously, some lockfile generators completely ignored the filename provided and just called the relevant tool in the directory, leading to some unexpected successes.

Testing

The behavior before this change:

❯ phylum --version
phylum v6.0.1

❯ cat dev-requirements.txt
pyyaml

❯ phylum parse dev-requirements.txt
❗ Error: could not parse dependency file "dev-requirements.txt"

Caused by:
    Failed to identify type for lockfile "dev-requirements.txt"

❌ 1 ❯ phylum parse -t pip dev-requirements.txt
❗ Error: could not parse dependency file "dev-requirements.txt"

Caused by:
    0: Failed to parse lockfile
    1: Failed to parse requirements file
    2: 0: at line 1, in Tag:
       pyyaml
             ^

The behavior after this change:

❯ target/debug/phylum --version
phylum v6.0.1-27-gacb4869

❯ cat dev-requirements.txt
pyyaml

❯ target/debug/phylum parse dev-requirements.txt

❗ Could not parse manifest: Parsing "dev-requirements.txt" requires a type to be specified

❌ 21 ❯ target/debug/phylum parse -t pip dev-requirements.txt
Generating lockfile for manifest "dev-requirements.txt" using Pip…
[
  {
    "name": "PyYAML",
    "version": "6.0.1",
    "type": "pypi",
    "lockfile": "dev-requirements.txt"
  }
]

❯ target/debug/phylum parse -t msbuild dev-requirements.txt
❗ Error: Failed to parse lockfile

Caused by:
    0: Syntax: 1:1 Unexpected characters outside the root element: p
    1: 1:1 Unexpected characters outside the root element: p

❗ Error: Could not parse dependency file "dev-requirements.txt" as "msbuild" type

Caused by:
    Dependency file parsing failed

❌ 1 ❯ target/debug/phylum parse -t npm dev-requirements.txt
Generating lockfile for manifest "dev-requirements.txt" using Npm…
❗ Error: Failed to parse lockfile

Caused by:
    expected value at line 1 column 1

❗ Error: Could not parse dependency file "dev-requirements.txt" as "npm" type

Caused by:
    Dependency file parsing failed