This change allows for parsing manifest files with names that don't
match the expected standards. For example, pip manifest files can be
named anything and they are therefore hard to identify. Pip lockfiles
can also be named anything but they will at least get picked up by the
fallback approach of attempting to parse with all known parsers
(assuming a "true" lockfile with no loose requirement entries).
A new custom error (UnknownManifestFormat) and exit code (21) were
created to be used when a dependency file is provided without a file
type and does not parse as a known lockfile, making it a likely manifest
file.
A change was made to report errors with the pip version check that
happens in the pip lockfile generator.
Additionally, a change was made to ensure ManifestWithoutGeneration
errors use the display path instead of the full path.
Prerequisite checks were also added for lockfile generation to account
for known manifest file names. Previously, some lockfile generators
completely ignored the filename provided and just called the relevant
tool in the directory, leading to some unexpected successes.
Testing
The behavior before this change:
❯ phylum --version
phylum v6.0.1
❯ cat dev-requirements.txt
pyyaml
❯ phylum parse dev-requirements.txt
❗ Error: could not parse dependency file "dev-requirements.txt"
Caused by:
Failed to identify type for lockfile "dev-requirements.txt"
❌ 1 ❯ phylum parse -t pip dev-requirements.txt
❗ Error: could not parse dependency file "dev-requirements.txt"
Caused by:
0: Failed to parse lockfile
1: Failed to parse requirements file
2: 0: at line 1, in Tag:
pyyaml
^
The behavior after this change:
❯ target/debug/phylum --version
phylum v6.0.1-27-gacb4869
❯ cat dev-requirements.txt
pyyaml
❯ target/debug/phylum parse dev-requirements.txt
❗ Could not parse manifest: Parsing "dev-requirements.txt" requires a type to be specified
❌ 21 ❯ target/debug/phylum parse -t pip dev-requirements.txt
Generating lockfile for manifest "dev-requirements.txt" using Pip…
[
{
"name": "PyYAML",
"version": "6.0.1",
"type": "pypi",
"lockfile": "dev-requirements.txt"
}
]
❯ target/debug/phylum parse -t msbuild dev-requirements.txt
❗ Error: Failed to parse lockfile
Caused by:
0: Syntax: 1:1 Unexpected characters outside the root element: p
1: 1:1 Unexpected characters outside the root element: p
❗ Error: Could not parse dependency file "dev-requirements.txt" as "msbuild" type
Caused by:
Dependency file parsing failed
❌ 1 ❯ target/debug/phylum parse -t npm dev-requirements.txt
Generating lockfile for manifest "dev-requirements.txt" using Npm…
❗ Error: Failed to parse lockfile
Caused by:
expected value at line 1 column 1
❗ Error: Could not parse dependency file "dev-requirements.txt" as "npm" type
Caused by:
Dependency file parsing failed
This change allows for parsing manifest files with names that don't match the expected standards. For example,
pip
manifest files can be named anything and they are therefore hard to identify. Pip lockfiles can also be named anything but they will at least get picked up by the fallback approach of attempting to parse with all known parsers (assuming a "true" lockfile with no loose requirement entries).A new custom error (
UnknownManifestFormat
) and exit code (21) were created to be used when a dependency file is provided without a file type and does not parse as a known lockfile, making it a likely manifest file.A change was made to report errors with the
pip
version check that happens in thepip
lockfile generator.Additionally, a change was made to ensure
ManifestWithoutGeneration
errors use the display path instead of the full path.Prerequisite checks were also added for lockfile generation to account for known manifest file names. Previously, some lockfile generators completely ignored the filename provided and just called the relevant tool in the directory, leading to some unexpected successes.
Testing
The behavior before this change:
The behavior after this change: