search

tern-tools / tern

Tern is a software composition analysis tool and Python library that generates a Software Bill of Materials for container images and Dockerfiles. The SBOM that Tern generates will give you a layer-by-layer view of what's inside your container in a variety of formats including human-readable, JSON, HTML, SPDX and more.
BSD 2-Clause "Simplified" License
960 stars 188 forks source link

license-expression does not handle invalid license key characters #1199

Closed rnjudge closed 1 year ago

rnjudge commented 1 year ago

Describe the bug When a license is reported with invalid license keys (i.e. anything besides letters and numbers, underscore, dot, colon or hyphen signs and spaces) the is_spdx_license_expression() function fails because the liense-expression library fails to handle the error.

To Reproduce tern report -i debian:latest -f spdxtagvalue -o debian.spdx

Error in terminal

2022-12-15 13:35:22,056 - DEBUG - __main__ - Starting...
2022-12-15 13:35:22,056 - DEBUG - prep - Setting up...
2022-12-15 13:35:22,293 - DEBUG - rootfs - Running command: sudo chmod +x /home/rose/rel-2.11/lib/python3.10/site-packages/tern/tools/fs_hash.sh
2022-12-15 13:35:37,321 - DEBUG - run - Starting analysis...
2022-12-15 13:35:37,321 - DEBUG - skopeo - Attempting to pull image "debian:latest"
2022-12-15 13:35:37,321 - DEBUG - rootfs - Running command: skopeo copy docker://debian:latest dir:/home/rose/.tern/temp
2022-12-15 13:35:41,367 - DEBUG - skopeo - Inspecting remote image "debian:latest"
2022-12-15 13:35:41,367 - DEBUG - rootfs - Running command: skopeo inspect docker://debian:latest
2022-12-15 13:35:45,238 - DEBUG - rootfs - Running command: tar -tf /home/rose/.tern/temp/4948a51a9a3f176f30ac619014f4a2da453a943244eacb53096ee9742eb7cef1
2022-12-15 13:35:45,952 - DEBUG - rootfs - Running command: tar -x -f /home/rose/.tern/temp/4948a51a9a3f176f30ac619014f4a2da453a943244eacb53096ee9742eb7cef1 -C /home/rose/.tern/temp/1/contents
2022-12-15 13:35:46,686 - DEBUG - rootfs - Running command: sudo /home/rose/rel-2.11/lib/python3.10/site-packages/tern/tools/fs_hash.sh /home/rose/.tern/temp/1/contents
2022-12-15 13:36:09,906 - DEBUG - common - Loading packages from cache: layer "2852aecdc1"
2022-12-15 13:36:09,949 - DEBUG - common - Loading files from cache: layer "2852aecdc1"
2022-12-15 13:36:11,070 - DEBUG - generator - Generating SPDX document...
Traceback (most recent call last):
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/license_expression/__init__.py", line 777, in validate
    parsed_expression = self.parse(expression, strict=strict)
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/license_expression/__init__.py", line 539, in parse
    tokens = list(self.tokenize(
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/license_expression/__init__.py", line 603, in tokenize
    for token in tokens:
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/license_expression/__init__.py", line 996, in replace_with_subexpression_by_license_symbol
    for token_group in token_groups:
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/license_expression/__init__.py", line 935, in build_token_groups_for_with_subexpression
    tokens = list(tokens)
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/license_expression/__init__.py", line 597, in <genexpr>
    tokens = (t for t in tokens if t.string and t.string.strip())
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/license_expression/__init__.py", line 921, in build_symbols_from_unknown_tokens
    for symtok in build_token_with_symbol():
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/license_expression/__init__.py", line 901, in build_token_with_symbol
    toksym = LicenseSymbol(string)
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/license_expression/__init__.py", line 1213, in __init__
    raise ExpressionError(
license_expression.ExpressionError: Invalid license key: the valid characters are: letters and numbers, underscore, dot, colon or hyphen signs and spaces: 'MIT/X11'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/rose/rel-2.11/bin/tern", line 8, in <module>
    sys.exit(main())
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/__main__.py", line 311, in main
    do_main(args)
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/__main__.py", line 123, in do_main
    crun.execute_image(args)
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/analyze/default/container/run.py", line 82, in execute_image
    report.report_out(args, full_image)
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/report/report.py", line 86, in report_out
    report = generate_report(args, *images)
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/report/report.py", line 41, in generate_report
    return generate_format(
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/report/report.py", line 56, in generate_format
    return mgr.driver.generate(images, print_inclusive)
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/formats/spdx/spdxtagvalue/generator.py", line 141, in generate
    report += mhelpers.get_image_block(image_obj, template) + '\n'
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/formats/spdx/spdxtagvalue/image_helpers.py", line 134, in get_image_block
    pkg_block = get_image_packages_block(image_obj, template)
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/formats/spdx/spdxtagvalue/image_helpers.py", line 37, in get_image_packages_block
    block += phelpers.get_package_block(package,
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/formats/spdx/spdxtagvalue/package_helpers.py", line 105, in get_package_block
    block += 'PackageLicenseDeclared: ' + spdx_common.get_package_license_declared(
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/formats/spdx/spdx_common.py", line 60, in get_package_license_declared
    if is_spdx_license_expression(package_license_declared):
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/formats/spdx/spdx_common.py", line 53, in is_spdx_license_expression
    return licensing.validate(license_data).errors == []
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/license_expression/__init__.py", line 780, in validate
    expression_info.invalid_symbols.append(e.token_string)
AttributeError: 'ExpressionError' object has no attribute 'token_string'

Expected behavior SPDX report should be generated.

rnjudge commented 1 year ago

I opened an issue for this with the license-expression library: https://github.com/nexB/license-expression/issues/76

I will create a workaround for this in the meantime.