Closed mymichu closed 2 weeks ago
Thanks for the detailed write up @mymichu - let me take a look today and see if we can get a patch released for this bug
Looks like our poetryPackage
needs a field to be able to track when pacakge.dependencies
is not x = "version"
and instead
x = [
version, markers
]
Let me get see what a fix looks like for this and get that pushed.
Are there any other fields we're missing that can be two different types that we need to consider? I'm having trouble tracking down an exact specification for the lockfile: https://github.com/anchore/syft/blob/ca0cc52d47b103642c4b8516cf7a09a7a2671656/syft/pkg/cataloger/python/parse_poetry_lock.go#L29-L38
@spiffcs Just FYI - not trying to take over the thread, but I think #2947 (which I filed) overlaps with this
The fix for this might take more than a day - I'm currently evaluating a new toml parser for syft. The current one doesn't give us the hooks we're looking for to customize this unmarshal function the way we want.
Here is where the error originates from when we call tree.Unmarshal.
I tried initially to change the dependency type to any
and doing a custom unmarshal given that it can be one of two things:
dependency = "version_constraint" ---> map[string]string
dependency = [
{},
{},
] ------> map[string][]ComplexVersion
I tried a couple solutions to get a custom Unmarshal
function hooked into the poetryPackages
type but found myself fighting with the library we use too much. It also looks like it removed support for this in its latest version:
https://github.com/pelletier/go-toml?tab=readme-ov-file#support-for-tomlunmarshaler-has-been-dropped
Currently I'm trying to get things working again with: https://github.com/BurntSushi/toml
This gives us a lot more flexibility where we can define a custom UnMarshalTOML
function for our poetry lock file type to get the inital decode done correctly:
https://godocs.io/github.com/BurntSushi/toml#example-package-UnmarshalTOML
@spiffcs Great work. Thank you for your quick response, support, and help. We appreciate this.
What happened: We have updated Syft from Version 1.5.0 to 1.6.0 and discovered that it has issues scanning certain poetry.lock files. We try to scan a poetry project with the syft 1.6.0 with executing the following command:
and then we discovered that the output of the console has the following warning:
and the json looks like followed:
With Syft version 1.5.0, it was possible to scan the lock file without any issues. If we remove the following part (see below) from the lock file then Syft 1.6.0 works. We can not do that because other teams are maintaining the lock files.
What you expected to happen:
I would expect that syft 1.6.0 has the same behaviour as the syft 1.5.0.
Steps to reproduce the issue:
Create poetry lock file and add the following snippet:
and execute the following command within the poetry project:
Anything else we need to know?:
Currently nothing
Environment:
syft version
: 1.6cat /etc/os-release
or similar): macOS 14.5