search

tern-tools / tern

Tern is a software composition analysis tool and Python library that generates a Software Bill of Materials for container images and Dockerfiles. The SBOM that Tern generates will give you a layer-by-layer view of what's inside your container in a variety of formats including human-readable, JSON, HTML, SPDX and more.
BSD 2-Clause "Simplified" License
963 stars 188 forks source link

Can't generate html output when running with scancode #844

Closed aalexanderr closed 3 years ago

aalexanderr commented 3 years ago

Describe the bug Can't generate html output when running tern with scancode:

To Reproduce Steps to reproduce the behavior:

  1. git clone "https://gerrit.onap.org/r/integration/docker/onap-python"
  2. cd onap-python
  3. either:
    1. docker build -t onappython - < Dockerfile && tern report -x scancode -f html -o report.html -i onappython:latest
    2. tern report -x scancode -f html -o report.html -d Dockerfile
  4. See error

Error in terminal

2020-12-15 22:22:03,866 - DEBUG - executor - Collecting file data...
2020-12-15 22:22:04,104 - DEBUG - generator - Creating HTML report...
Traceback (most recent call last):
  File "/home/vagrant/ternenv/bin/tern", line 8, in <module>
    sys.exit(main())
  File "/home/vagrant/ternenv/lib/python3.6/site-packages/tern/__main__.py", line 192, in main
    do_main(args)
  File "/home/vagrant/ternenv/lib/python3.6/site-packages/tern/__main__.py", line 95, in do_main
    crun.execute_image(args)
  File "/home/vagrant/ternenv/lib/python3.6/site-packages/tern/analyze/default/container/run.py", line 87, in execute_image
    report.report_out(args, full_image)
  File "/home/vagrant/ternenv/lib/python3.6/site-packages/tern/report/report.py", line 70, in report_out
    report = generate_report(args, *images)
  File "/home/vagrant/ternenv/lib/python3.6/site-packages/tern/report/report.py", line 50, in generate_report
    return generate_format(images, args.report_format)
  File "/home/vagrant/ternenv/lib/python3.6/site-packages/tern/report/report.py", line 64, in generate_format
    return mgr.driver.generate(images)
  File "/home/vagrant/ternenv/lib/python3.6/site-packages/tern/formats/html/generator.py", line 299, in generate
    report = create_html_report(report_dict, image_obj_list)
  File "/home/vagrant/ternenv/lib/python3.6/site-packages/tern/formats/html/generator.py", line 278, in create_html_report
    report = report + '\n' + write_licenses(image_obj_list)
  File "/home/vagrant/ternenv/lib/python3.6/site-packages/tern/formats/html/generator.py", line 256, in write_licenses
    licenses = get_licenses_only(image_obj_list)
  File "/home/vagrant/ternenv/lib/python3.6/site-packages/tern/report/content.py", line 40, in get_licenses_only
    pkg_licenses = get_layer_packages_licenses(layer)
  File "/home/vagrant/ternenv/lib/python3.6/site-packages/tern/report/content.py", line 19, in get_layer_packages_licenses
    package_licenses = get_package_licenses(package)
  File "/home/vagrant/ternenv/lib/python3.6/site-packages/tern/report/content.py", line 54, in get_package_licenses
    pkg_licenses.add(package.pkg_license)
TypeError: unhashable type: 'dict'

Expected behavior Get a html report

Environment you are running Tern on Enter all that apply

Please attach files if they exist

rnjudge commented 3 years ago

@abhaykatheria do you want to take a look at this?

abhaykatheria commented 3 years ago

Oh ok cool.

abhaykatheria commented 3 years ago

@aalexanderr @rnjudge I ran tern on the given image using same commands(BOTH OF THEM) and it ran well. Here's the report I got. report.html

And these are the logs

tern report -x scancode -f html -o report.html -d Dockerfile log-1 tern.log

docker build -t onappython - < Dockerfile && tern report -x scancode -f html -o report.html -i onappython:latest log-2 tern.log

@aalexanderr I would recommend updating tern and scancode-toolkit.

aalexanderr commented 3 years ago

@abhaykatheria What version of tern did you use? The html report links to this commit: https://github.com/tern-tools/tern/commit/c8f336a7137e0f34d7cb7a55ee4fcb9cf104b7b4 but it shows me 404.

I've checked your fork but the only active branch I see (master) is the same as here.

Both tern and scancode I used are latest releases. Same with python- latest release of 3.6.

$ scancode --version
ScanCode version 3.2.3
Tern version:
$ tern --version
Tern version 2.3.0
python version = 3.6.12 (default, Aug 17 2020, 23:45:20)

(before submitting I have verifed it on 2 different host machines with different vagrant versions and two different versions of generic/ubuntu2004 versions (latest and a bit older one).

abhaykatheria commented 3 years ago

I installed it manually from the source code (Instructions are in contribution.md). I didn't made any commits I checked the git logs. its strange the commit hash in report is pointing to nowhere. @rnjudge can you take a look at this ..?

aalexanderr commented 3 years ago

I did everything as in linked contributing, then pip3 install scancode-toolkit[full] and went with steps to reproduce. It resulted in the same error as initially reported. BTW the unknown hash is the last commit from the onap-python repo :)\ Additionally, the report you've linked has no licenses listed :(

rnjudge commented 3 years ago

Bumping the priority on this. Will work on a fix in the next week or two. @abhaykatheria if you have time, would be lovely for you to take a look also ;) We have two users reporting issues with the html format when using scancode.

rnjudge commented 3 years ago

@aalexanderr Sincere apologies for the delay on this fix. Should work now.