tern-tools / tern

Tern is a software composition analysis tool and Python library that generates a Software Bill of Materials for container images and Dockerfiles. The SBOM that Tern generates will give you a layer-by-layer view of what's inside your container in a variety of formats including human-readable, JSON, HTML, SPDX and more.
BSD 2-Clause "Simplified" License
967 stars 188 forks source link

Scan failing with 2.9.0 release on .split #1101

Closed jasona7 closed 2 years ago

jasona7 commented 2 years ago

Describe the bug A clear and concise description of what the bug is.

We run weekly Tern scans against hundreds of images on a Harbor registry (v2.1.3) with the previous Tern release v2.8.0. We upgraded to the v2.9.0 release and are experiencing Traceback split errors from /usr/local/lib/python3.9/re.py. Not all scans are failing, about 40% of the scans fail.

To Reproduce Steps to reproduce the behavior:

  1. Execute:

sudo ./docker_run.sh tern-debug "report -i %REGISTRY%/library/fluentd-syslog:v2"

  1. See error 2022-01-06 13:48:11,205 - DEBUG - rootfs - Running command: skopeo copy docker://%REGISTRY%/library/fluentd-syslog:v2 dir:/root/.tern/temp Traceback (most recent call last): File "/usr/local/bin/tern", line 8, in sys.exit(main()) File "/usr/local/lib/python3.9/site-packages/tern/main.py", line 286, in main do_main(args) File "/usr/local/lib/python3.9/site-packages/tern/main.py", line 116, in do_main crun.execute_image(args) File "/usr/local/lib/python3.9/site-packages/tern/analyze/default/container/run.py", line 70, in execute_image full_image = cimage.load_full_image( File "/usr/local/lib/python3.9/site-packages/tern/analyze/default/container/image.py", line 39, in load_full_image image.load_image(load_until_layer) File "/usr/local/lib/python3.9/site-packages/tern/classes/oci_image.py", line 105, in load_image repo_dict = general.parse_image_string( File "/usr/local/lib/python3.9/site-packages/tern/utils/general.py", line 296, in parse_image_string tokens = re.split(r'[@:]', image_string) File "/usr/local/lib/python3.9/re.py", line 231, in split return _compile(pattern, flags).split(string, maxsplit) TypeError: expected string or bytes-like object

Expected behavior Successful Scan

docker pull & inspect using the same image works

Environment you are running Tern on

vEpiphyte commented 2 years ago

I've also experienced this with Python 3.10.1 and tern 2.9.0.

$ tern report -i vertexproject/synapse-base-image3:py38
2022-01-18 14:44:53,239 - DEBUG - __main__ - Starting...
2022-01-18 14:44:53,240 - DEBUG - prep - Setting up...
2022-01-18 14:44:53,284 - DEBUG - rootfs - Running command: sudo chmod +x /home/user/.pyenv/versions/3.10.1/envs/fp3101/lib/python3.10/site-packages/tern/tools/fs_hash.sh
2022-01-18 14:44:53,288 - DEBUG - run - Starting analysis...
2022-01-18 14:44:53,289 - DEBUG - skopeo - Attempting to pull image "vertexproject/synapse-base-image3:py38"
2022-01-18 14:44:53,289 - DEBUG - rootfs - Running command: skopeo copy docker://vertexproject/synapse-base-image3:py38 dir:/home/user/.tern/temp
Traceback (most recent call last):
  File "/home/user/.pyenv/versions/3.10.1/envs/fp3101/bin/tern", line 8, in <module>
    sys.exit(main())
  File "/home/user/.pyenv/versions/3.10.1/envs/fp3101/lib/python3.10/site-packages/tern/__main__.py", line 286, in main
    do_main(args)
  File "/home/user/.pyenv/versions/3.10.1/envs/fp3101/lib/python3.10/site-packages/tern/__main__.py", line 116, in do_main
    crun.execute_image(args)
  File "/home/user/.pyenv/versions/3.10.1/envs/fp3101/lib/python3.10/site-packages/tern/analyze/default/container/run.py", line 70, in execute_image
    full_image = cimage.load_full_image(
  File "/home/user/.pyenv/versions/3.10.1/envs/fp3101/lib/python3.10/site-packages/tern/analyze/default/container/image.py", line 39, in load_full_image
    image.load_image(load_until_layer)
  File "/home/user/.pyenv/versions/3.10.1/envs/fp3101/lib/python3.10/site-packages/tern/classes/oci_image.py", line 105, in load_image
    repo_dict = general.parse_image_string(
  File "/home/user/.pyenv/versions/3.10.1/envs/fp3101/lib/python3.10/site-packages/tern/utils/general.py", line 296, in parse_image_string
    tokens = re.split(r'[@:]', image_string)
  File "/home/user/.pyenv/versions/3.10.1/lib/python3.10/re.py", line 230, in split
    return _compile(pattern, flags).split(string, maxsplit)
TypeError: expected string or bytes-like object
nishakm commented 2 years ago

I'm able to reproduce this. A fix is coming up, followed by a 2.9.1 release once the bug fixes are merged.