openshift-eng / elliott

Apache License 2.0
15 stars 28 forks source link

Halt if rpm CVEs don't have corresponding build #367

Closed joepvd closed 2 years ago

joepvd commented 2 years ago

Throw an error when there are CVE tracker bugs that cannot get attached because the component that they are about is not getting shipped.

Currently an event like this will go unnoticed, as the log is not scrutinized.

The can be bypassed by either fixing the issue, or by exclkuding the relevant bugs from the assembly:

releases.<name>.issues.exclude: [1234]

openshift-bot commented 2 years ago

Build #1

GLOB sdist-make: /mnt/workspace/jenkins/working/art-tools_elliott_PR-367/setup.py
py36 create: /mnt/workspace/jenkins/working/art-tools_elliott_PR-367/.tox/py36
py36 installdeps: -rrequirements-dev.txt
py36 inst: /mnt/workspace/jenkins/working/art-tools_elliott_PR-367/.tox/.tmp/package/1/rh-elliott-2.0.14.dev54+ga7dc16e.zip
py36 installed: aiodns==3.0.0,aiohttp==3.8.1,aiosignal==1.2.0,appdirs==1.4.4,astroid==2.11.7,async-timeout==4.0.2,asynctest==0.13.0,attrs==21.4.0,Brotli==1.0.9,cached-property==1.5.2,cchardet==2.1.7,certifi==2022.6.15,cffi==1.15.1,charset-normalizer==2.0.12,click==8.0.4,contextvars==2.4,coverage==5.5,cryptography==37.0.4,dataclasses==0.8,decorator==5.1.1,defusedxml==0.7.1,dill==0.3.4,distlib==0.3.5,errata-tool==1.27.1,filelock==3.4.1,flake8==3.8.4,flexmock==0.11.3,frozenlist==1.2.0,future==0.18.2,gssapi==1.7.3,idna==3.3,idna-ssl==1.1.0,immutables==0.18,importlib-metadata==4.8.3,importlib-resources==5.4.0,iniconfig==1.1.1,isort==5.10.1,jeepney==0.7.1,jira==3.2.0,jsonpath-rw==1.4.0,keyring==23.4.1,koji==1.29.1,krb5==0.3.0,lazy-object-proxy==1.7.1,mccabe==0.6.1,mock==4.0.3,multidict==5.2.0,mypy==0.971,mypy-extensions==0.4.3,oauthlib==3.2.0,packaging==21.3,platformdirs==2.4.0,pluggy==1.0.0,ply==3.11,py==1.11.0,pycares==4.2.1,pycodestyle==2.6.0,pycparser==2.21,pyflakes==2.2.0,pygit2==1.6.1,pylint==2.13.9,pyparsing==3.0.9,pyspnego==0.5.3,pytest==6.2.5,python-bugzilla==3.2.0,python-dateutil==2.8.2,PyYAML==6.0,requests==2.27.1,requests-gssapi==1.2.3,requests-kerberos==0.14.0,requests-oauthlib==1.3.1,requests-toolbelt==0.9.1,rh-elliott @ file:///mnt/workspace/jenkins/working/art-tools_elliott_PR-367/.tox/.tmp/package/1/rh-elliott-2.0.14.dev54%2Bga7dc16e.zip,ruamel.yaml==0.17.21,ruamel.yaml.clib==0.2.6,SecretStorage==3.3.2,semver==2.13.0,setuptools-scm==6.4.2,six==1.16.0,tenacity==8.0.1,toml==0.10.2,tomli==1.2.3,tox==3.20.1,typed-ast==1.5.4,typing-extensions==4.1.1,urllib3==1.26.11,virtualenv==20.2.2,wrapt==1.14.1,yarl==1.7.2,zipp==3.6.0
py36 run-test-pre: PYTHONHASHSEED='2050298843'
py36 run-test: commands[0] | coverage run --branch --source elliottlib -m unittest discover -t . -s tests/
/mnt/workspace/jenkins/working/art-tools_elliott_PR-367/.tox/py36/lib/python3.6/site-packages/spnego/_ntlm_raw/crypto.py:22: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release.
  from cryptography.hazmat.backends import default_backend
...........For flaw bug 5 - these tracker bugs do not have a valid whiteboard component value: [1] Cannot reliably determine if flaw bug is first fix. Check tracker bugs manually
......................................................changed 123 from status1 to status2
.123 is already on status1
.changed 123 from status1 to status2
..changed 123 from status1 to status2
.123 is already on status1
.changed 123 from status1 to status2
................../mnt/workspace/jenkins/working/art-tools_elliott_PR-367/.tox/py36/lib64/python3.6/site-packages/aiohttp/connector.py:777: DeprecationWarning: The object should be created within an async function
  loop=loop,
/mnt/workspace/jenkins/working/art-tools_elliott_PR-367/.tox/py36/lib64/python3.6/site-packages/aiohttp/connector.py:782: DeprecationWarning: The object should be created within an async function
  resolver = DefaultResolver(loop=self._loop)
........./mnt/workspace/jenkins/working/art-tools_elliott_PR-367/elliottlib/errata_async.py:41: RuntimeWarning: coroutine 'AsyncMockMixin._execute_mock_call' was never awaited
  resp.raise_for_status()
........................................................Retrieving bugs for advisory ['12345']
...
----------------------------------------------------------------------
Ran 157 tests in 1.429s

OK
py36 run-test: commands[1] | coverage report
Name                                              Stmts   Miss Branch BrPart  Cover
-----------------------------------------------------------------------------------
elliottlib/__init__.py                                8      2      2      1    70%
elliottlib/assembly.py                               92      9     54      7    89%
elliottlib/assertion.py                              11      0      6      0   100%
elliottlib/brew.py                                  317    188    134      5    35%
elliottlib/bug/__init__.py                            0      0      0      0   100%
elliottlib/build_finder.py                          114     12     74     15    86%
elliottlib/bzutil.py                                666    208    310     38    66%
elliottlib/cincinnati.py                             21     21      4      0     0%
elliottlib/cli/__init__.py                            0      0      0      0   100%
elliottlib/cli/__main__.py                          268    268     72      0     0%
elliottlib/cli/add_metadata_cli.py                   32     32      4      0     0%
elliottlib/cli/advisory_commons_cli.py               89     89     38      0     0%
elliottlib/cli/advisory_drop_cli.py                  20     20      2      0     0%
elliottlib/cli/advisory_images_cli.py                13     13      2      0     0%
elliottlib/cli/advisory_impetus_cli.py               21     21      6      0     0%
elliottlib/cli/attach_bugs_cli.py                    39     39     16      0     0%
elliottlib/cli/attach_cve_flaws_cli.py              116     63     48      6    43%
elliottlib/cli/change_state_cli.py                   52     52     20      0     0%
elliottlib/cli/cli_opts.py                           13      0     15      0   100%
elliottlib/cli/common.py                             49     14      6      1    65%
elliottlib/cli/create_cli.py                         70     70     20      0     0%
elliottlib/cli/create_placeholder_cli.py             40     40     12      0     0%
elliottlib/cli/create_textonly_cli.py                56     56      8      0     0%
elliottlib/cli/find_bugs_blocker_cli.py              43      4     10      3    87%
elliottlib/cli/find_bugs_qe_cli.py                   36      4      6      0    90%
elliottlib/cli/find_bugs_sweep_cli.py               164     50     74     11    64%
elliottlib/cli/find_builds_cli.py                   255    188    169      2    20%
elliottlib/cli/get_golang_versions_cli.py            55     21     32      5    49%
elliottlib/cli/list_cli.py                           16     16      4      0     0%
elliottlib/cli/puddle_advisories_cli.py              36     36     10      0     0%
elliottlib/cli/remove_bugs_cli.py                    50      9     16      5    79%
elliottlib/cli/repair_bugs_cli.py                    64     10     28      7    77%
elliottlib/cli/rhcos_cli.py                          96     96     54      0     0%
elliottlib/cli/rpmdiff_cli.py                       100    100     34      0     0%
elliottlib/cli/tag_builds_cli.py                    128    128     62      0     0%
elliottlib/cli/tarball_sources_cli.py                81     81     26      0     0%
elliottlib/cli/validate_rhsa.py                      40     40     10      0     0%
elliottlib/cli/verify_attached_bugs_cli.py          202    145    121      1    20%
elliottlib/cli/verify_attached_operators_cli.py     114     81     52      0    20%
elliottlib/cli/verify_cvp_cli.py                    286    286    138      0     0%
elliottlib/constants.py                              50      0      0      0   100%
elliottlib/dotconfig.py                              54     11     30      6    75%
elliottlib/errata.py                                308    193    134      4    34%
elliottlib/errata_async.py                          109      6     38      5    93%
elliottlib/exceptions.py                             12      0      0      0   100%
elliottlib/exectools.py                              75      5     24      3    90%
elliottlib/gitdata.py                               170    137     76      0    13%
elliottlib/imagecfg.py                               18      2      0      0    89%
elliottlib/logutil.py                                 9      1      2      1    82%
elliottlib/metadata.py                              154     43     76     18    67%
elliottlib/model.py                                 102     22     30      2    79%
elliottlib/openshiftclient.py                        34     34      8      0     0%
elliottlib/pushd.py                                  20      0      2      0   100%
elliottlib/resultsdb.py                              28     28      8      0     0%
elliottlib/rhcos.py                                  51     51     12      0     0%
elliottlib/rpm_utils.py                              30     14     16      5    46%
elliottlib/rpmcfg.py                                  5      2      0      0    60%
elliottlib/rpmdiff.py                                31      0      0      0   100%
elliottlib/runtime.py                               269    200    104      1    20%
elliottlib/tarball_sources.py                        90     61     34      0    30%
elliottlib/util.py                                  299    166    110      9    41%
-----------------------------------------------------------------------------------
TOTAL                                              5791   3488   2403    161    37%
py36 run-test: commands[2] | flake8
___________________________________ summary ____________________________________
  py36: commands succeeded
  congratulations :)