Open pdxjohnny opened 4 years ago
This would probably help us figure out what's wrong with #737
.nreqs
format which is a combinaion of rpmThe idea behind the work that was done so far in the above branch was to produce the full dependency tree for a given python package.
-j, --json Display dependency tree as json. This will yield "raw"
output that may be used by external tools. This option
overrides all other options.
$ git clone https://github.com/intel/dffml
$ cd dffml
$ python -m venv .venv
$ git checkout -b deptree
$ . .venv/Scripts/activate
$ pip install -e .[dev]
$ cd examples/shouldi
$ pip install -e .[dev]
ActivateVirtualEnvCommand
into something that doesn't mess with os.environ
and behaves more like mkvenv()
(https://github.com/intel/dffml/tree/main/dffml/util/testing/consoletest/)dffml.run_command()/subprocess.check_call()
A discussion today revealed we should go with the file paring approach (rather than pipdeptree
)
$ sbom4python --format json --sbom spdx -m black
{
"SPDXID": "SPDXRef-DOCUMENT",
"spdxVersion": "SPDX-2.2",
"creationInfo": {
"comment": "This document has been automatically generated.",
"creators": [
"Tool: sbom4python-0.7.0"
],
"created": "2023-03-02T13:53:48Z",
"licenseListVersion": "3.18"
},
"name": "black",
"dataLicense": "CC0-1.0",
"documentNamespace": "http://spdx.org/spdxdocs/black-7ebf1fb9-a781-41f5-9e57-a6bba6969ecb",
"packages": [
{
"SPDXID": "SPDXRef-Package-1-black",
"name": "black",
"versionInfo": "23.1.1.dev8+g25d886f",
"supplier": "Organization: ukasz Langa (lukasz@langa.pl)",
"downloadLocation": "NONE",
"filesAnalyzed": false,
"licenseConcluded": "MIT",
"licenseDeclared": "MIT",
"copyrightText": "NOASSERTION",
"externalRefs": [
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceLocator": "pkg:pypi/black@23.1.1.dev8+g25d886f",
"referenceType": "purl"
},
{
"referenceCategory": "SECURITY",
"referenceLocator": "cpe:2.3:a:ukasz_langa:black:23.1.1.dev8+g25d886f:*:*:*:*:*:*:*",
"referenceType": "cpe23Type"
}
]
},
{
"SPDXID": "SPDXRef-Package-2-click",
"name": "click",
"versionInfo": "8.1.3",
"supplier": "Organization: Armin Ronacher (armin.ronacher@active-4.com)",
"downloadLocation": "NONE",
"filesAnalyzed": false,
"licenseConcluded": "BSD-3-Clause",
"licenseDeclared": "BSD-3-Clause",
"copyrightText": "NOASSERTION",
"externalRefs": [
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceLocator": "pkg:pypi/click@8.1.3",
"referenceType": "purl"
},
{
"referenceCategory": "SECURITY",
"referenceLocator": "cpe:2.3:a:armin_ronacher:click:8.1.3:*:*:*:*:*:*:*",
"referenceType": "cpe23Type"
}
]
},
{
"SPDXID": "SPDXRef-Package-3-mypy-extensions",
"name": "mypy-extensions",
"versionInfo": "1.0.0",
"supplier": "Organization: The mypy developers (jukka.lehtosalo@iki.fi)",
"downloadLocation": "NONE",
"filesAnalyzed": false,
"licenseConcluded": "MIT",
"licenseDeclared": "MIT",
"copyrightText": "NOASSERTION",
"externalRefs": [ [97/296]
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceLocator": "pkg:pypi/black@23.1.1.dev8+g25d886f",
"referenceType": "purl"
},
{
"referenceCategory": "SECURITY",
"referenceLocator": "cpe:2.3:a:ukasz_langa:black:23.1.1.dev8+g25d886f:*:*:*:*:*:*:*",
"referenceType": "cpe23Type"
}
]
},
{
"SPDXID": "SPDXRef-Package-2-click",
"name": "click",
"versionInfo": "8.1.3",
"supplier": "Organization: Armin Ronacher (armin.ronacher@active-4.com)",
"downloadLocation": "NONE",
"filesAnalyzed": false,
"licenseConcluded": "BSD-3-Clause",
"licenseDeclared": "BSD-3-Clause",
"copyrightText": "NOASSERTION",
"externalRefs": [
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceLocator": "pkg:pypi/click@8.1.3",
"referenceType": "purl"
},
{
"referenceCategory": "SECURITY",
"referenceLocator": "cpe:2.3:a:armin_ronacher:click:8.1.3:*:*:*:*:*:*:*",
"referenceType": "cpe23Type"
}
]
},
{
"SPDXID": "SPDXRef-Package-3-mypy-extensions",
"name": "mypy-extensions",
"versionInfo": "1.0.0",
"supplier": "Organization: The mypy developers (jukka.lehtosalo@iki.fi)",
"downloadLocation": "NONE",
"filesAnalyzed": false,
"licenseConcluded": "MIT",
"licenseDeclared": "MIT",
"copyrightText": "NOASSERTION",
"externalRefs": [
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceLocator": "pkg:pypi/mypy-extensions@1.0.0",
"referenceType": "purl"
},
{
"referenceCategory": "SECURITY",
"referenceLocator": "cpe:2.3:a:the_mypy_developers:mypy-extensions:1.0.0:*:*:*:*:*:*:*",
"referenceType": "cpe23Type"
}
]
},
{
"SPDXID": "SPDXRef-Package-4-packaging",
"name": "packaging",
"versionInfo": "23.0",
"supplier": "Organization: Donald Stufft (donald@stufft.io)",
"downloadLocation": "NONE",
"filesAnalyzed": false,
"licenseConcluded": "NOASSERTION",
"licenseDeclared": "NOASSERTION",
"copyrightText": "NOASSERTION",
"externalRefs": [
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceLocator": "pkg:pypi/packaging@23.0",
"referenceType": "purl"
},
{
"referenceCategory": "SECURITY",
"referenceLocator": "cpe:2.3:a:donald_stufft:packaging:23.0:*:*:*:*:*:*:*",
"referenceType": "cpe23Type"
}
]
},
{
"SPDXID": "SPDXRef-Package-5-pathspec",
"name": "pathspec",
"versionInfo": "0.11.0",
"supplier": "Organization: Caleb P. (cpburnz@gmail.com)",
"downloadLocation": "NONE",
"filesAnalyzed": false,
"licenseConcluded": "NOASSERTION",
"licenseDeclared": "NOASSERTION",
"copyrightText": "NOASSERTION",
"externalRefs": [
https://github.com/intel/dffml/commits/shouldi_dep_tree
The idea behind the work that was done so far in the above branch was to produce the full dependency tree for a given python package. We'll want this to be the
shouldi deptree
command (as opposed toshouldi install
)It's currently in a state where it can grab package names out of setup.py files. It also needs to be able to grab them out of
setup.cfg
files and requirements.txt files.Another thing that's missing is the version number of the package. Right now the latest version of each package is being downloaded. We need to check if the package was pinned to a version and download that version (if
example_package==0.3.1
for example we'd download version 0.3.1 of example_package). Come up with reasonable ways to handle all the following cases: https://stackoverflow.com/a/50842265/12310488There is also a new output operation that needs to be made for this work. It's the
Tree
output operation. It's not working at the moment. What it should do is output adict
similar to the way config structures work with their use ofplugin
andconfig
(you'll see what I'm about when you read the existing code.requirements-dev.txt
service/http/setup.cfg
https://github.com/intel/dffml/blob/0a2e053f5f8e361054f329a3f763982fb1e4d1f7/examples/shouldi/tests/test_dep_tree.py#L1-L169