opensbom-generator / spdx-sbom-generator

Support CI generation of SBOMs via golang tooling.
406 stars 110 forks source link

SPDX SBOM Generator Bug Report - Plugin pipenv return error failed to read modules #228

Open PeterDaveHello opened 3 years ago

PeterDaveHello commented 3 years ago

Summary

Got error message Unable to fetch package details when I try to generate the SBOM, the error has no more details or verbose log, I have no idea what's happening or what's required to make it work.

$ spdx-sbom-generator 
INFO[2021-08-16T17:10:17+08:00] Starting to generate SPDX ...                
INFO[2021-08-16T17:10:17+08:00] Running generator for Module Manager: `pipenv` with output `bom-pipenv.spdx` 
INFO[2021-08-16T17:10:17+08:00] Current Language Version Python 3.7.5        
ERRO[2021-08-16T17:10:18+08:00] Unable to fetch package details              
INFO[2021-08-16T17:10:18+08:00] Command has completed with errors for some package managers, see details below 
INFO[2021-08-16T17:10:18+08:00] Plugin pipenv return error failed to read modules 

Background

Environment:

Steps to get the problem:

  1. Download sbom-spdx-generator from https://github.com/spdx/spdx-sbom-generator/releases/tag/v0.0.13, extract the binary to $PATH
  2. Run sbom-spdx-generator binary in a private project path(It's a python project, using pipenv to help manage the packages)
  3. Observe the following error:
INFO[2021-08-16T17:10:17+08:00] Starting to generate SPDX ...                
INFO[2021-08-16T17:10:17+08:00] Running generator for Module Manager: `pipenv` with output `bom-pipenv.spdx` 
INFO[2021-08-16T17:10:17+08:00] Current Language Version Python 3.7.5        
ERRO[2021-08-16T17:10:18+08:00] Unable to fetch package details              
INFO[2021-08-16T17:10:18+08:00] Command has completed with errors for some package managers, see details below 
INFO[2021-08-16T17:10:18+08:00] Plugin pipenv return error failed to read modules 

Expected behavior

Expect to produce SBOM.

Screenshots

image

Repository

It's a private repository, but I might be able to provide the Pipfile file of pipenv, if it's something will help to reproduce the bug.

Acceptance Criteria

Attributus commented 2 years ago

I ran into this same issue. My resolution was to create a setup.cfg. In addition, I export out a requirements.txt using pipreqs to get all packages in .py files. From that I used pipenv to create the Pipfile & Pipfile.lock. This article helped by defining minimal package requirements for python: https://packaging-guide.openastronomy.org/en/latest/minimal.html

Prior to this I tried using pipenv to create Pipfile & Pipfile.lock, I tried using a requirements.txt alone, and then I tried a combination of these. None of these worked until I added the initial step of creating a minimal setup.cfg first.

Hope it helps!

MDanialSaleem commented 1 year ago

@Attributus thank you for your comment, saved me hours of ramming my head against the wall.

Exact steps I took to make this work in the project source dir:

python3 -m venv env
source env/bin/activate
python3 -m pip install -r requirements.txt

Created this setup.cfg file:

[metadata]
name = my-package
description = My package description
long_description = file: README.rst
author = Your Name
author_email = your@email.com
url = https://link-to-your-project
license = BSD 3-Clause License

[options]
zip_safe = False
packages = find:

Ran spdx-sbom-generator and it worked.

And now that I look into the code, I find this function:

func IsValidRootModule(path string) bool {
    modules := []string{manifestSetupCfg, manifestSetupPy}
    for i := range modules {
        if helper.Exists(filepath.Join(path, modules[i])) {
            return true
        }
    }
    return false
}

Which checks for setup.cfg/setup.py to establish if the directory is root module. I am not much of an expert in python dependency management, but it may be a good idea to better the error generation of sbom tool, the current errors is kind of cryptic. Happy to help with a PR, but would need some pointers as to what the expected behavior should be!