search

openshift-eng / elliott

Apache License 2.0
15 stars 28 forks source link

ART-6236: find-bugs:sweep: Add --cve-only option #525

Closed vfreex closed 1 year ago

vfreex commented 1 year ago

This feature is requested by OpenShift QE to find CVE trackers against a release branch.

openshift-ci-robot commented 1 year ago

@vfreex: This pull request references ART-6236 which is a valid jira issue.

In response to [this](https://github.com/openshift/elliott/pull/525): >This feature is requested by OpenShift QE to find CVE trackers against a release branch. Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
openshift-bot commented 1 year ago

Build #1

GLOB sdist-make: /mnt/workspace/jenkins/working/art-tools_elliott_PR-525/setup.py
py38 create: /mnt/workspace/jenkins/working/art-tools_elliott_PR-525/.tox/py38
py38 installdeps: -rrequirements-dev.txt, -rrequirements.txt
py38 inst: /mnt/workspace/jenkins/working/art-tools_elliott_PR-525/.tox/.tmp/package/1/rh-elliott-2.0.15.dev26+g621cb71.zip
py38 installed: aiodns==3.0.0,aiohttp==3.8.4,aiosignal==1.3.1,astroid==2.15.0,async-timeout==4.0.2,asynctest==0.13.0,attrs==22.2.0,Brotli==1.0.9,cachetools==5.3.0,cchardet==2.1.7,certifi==2022.12.7,cffi==1.15.1,chardet==5.1.0,charset-normalizer==3.1.0,click==8.1.3,colorama==0.4.6,coverage==7.2.1,decorator==5.1.1,defusedxml==0.7.1,dill==0.3.6,distlib==0.3.6,errata-tool==1.30.0,exceptiongroup==1.1.0,filelock==3.9.0,flake8==6.0.0,flexmock==0.11.3,frozenlist==1.3.3,future==0.18.3,gssapi==1.8.2,idna==3.4,iniconfig==2.0.0,isort==5.12.0,jira==3.4.1,jsonpath-rw==1.4.0,koji==1.32.0,lazy-object-proxy==1.9.0,mccabe==0.7.0,mock==5.0.1,multidict==6.0.4,mypy==1.1.1,mypy-extensions==1.0.0,oauthlib==3.2.2,packaging==23.0,platformdirs==3.1.0,pluggy==1.0.0,ply==3.11,pycares==4.3.0,pycodestyle==2.10.0,pycparser==2.21,pyflakes==3.0.1,pygit2==1.10.1,pylint==2.16.4,pyproject_api==1.5.0,pytest==7.2.2,python-bugzilla==3.2.0,python-dateutil==2.8.2,PyYAML==6.0,requests==2.28.2,requests-gssapi==1.2.3,requests-oauthlib==1.3.1,requests-toolbelt==0.10.1,rh-elliott @ file:///mnt/workspace/jenkins/working/art-tools_elliott_PR-525/.tox/.tmp/package/1/rh-elliott-2.0.15.dev26%2Bg621cb71.zip,ruamel.yaml==0.17.21,ruamel.yaml.clib==0.2.7,semver==2.13.0,setuptools-scm==7.1.0,six==1.16.0,tenacity==8.2.2,tomli==2.0.1,tomlkit==0.11.6,tox==4.4.6,typing_extensions==4.5.0,urllib3==1.26.14,virtualenv==20.20.0,wrapt==1.15.0,yarl==1.8.2
py38 run-test-pre: PYTHONHASHSEED='1029418740'
py38 run-test: commands[0] | coverage run --branch --source elliottlib -m unittest discover -t . -s tests/
.............................................Cannot find any corresponding flaw bugs for these trackers: [13]. These trackers do not have a valid whiteboard component value: [12].
..BZ 9 is ignored because its status was MODIFIED at the moment of sweep cutoff (2021-06-30 12:30:00), however its status changed back to ['ASSIGNED'] afterwards
.....................changed 123 from status1 to status2
.123 is already on status1
.changed 123 from status1 to status2
..changed 123 from status1 to status2
.123 is already on status1
.changed 123 from status1 to status2
.........................................F.F.......Bug(s) ['OCPBUGS-4', 'OCPBUGS-6', 'OCPBUGS-5'] have CVE in description but do not have TrackerBug labels. Please investigate.
.....E......................................F./mnt/workspace/jenkins/working/art-tools_elliott_PR-525/.tox/py38/lib64/python3.8/site-packages/aiohttp/client.py:342: ResourceWarning: Unclosed client session <aiohttp.client.ClientSession object at 0x7f0ed82719a0>
  _warnings.warn(
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7f0ed82719a0>
.
======================================================================
ERROR: test_find_bugs_mode_search (tests.test_find_bugs_sweep_cli.TestFindBugsMode)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/workspace/jenkins/working/art-tools_elliott_PR-525/.tox/py38/lib/python3.8/site-packages/mock/mock.py", line 1417, in patched
    return func(*newargs, **newkeywargs)
  File "/mnt/workspace/jenkins/working/art-tools_elliott_PR-525/tests/test_find_bugs_sweep_cli.py", line 28, in test_find_bugs_mode_search
    find_bugs = FindBugsMode(status=['foo', 'bar'])
TypeError: __init__() missing 1 required positional argument: 'cve_only'

======================================================================
FAIL: test_find_bugs_blocker (tests.test_find_bugs_blocker_cli.FindBugsBlockerTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/workspace/jenkins/working/art-tools_elliott_PR-525/tests/test_find_bugs_blocker_cli.py", line 41, in test_find_bugs_blocker
    self.fail(t)
AssertionError: Traceback (most recent call last):

  File "/mnt/workspace/jenkins/working/art-tools_elliott_PR-525/.tox/py38/lib/python3.8/site-packages/click/testing.py", line 408, in invoke
    return_value = cli.main(args=args or (), prog_name=prog_name, **extra)

  File "/mnt/workspace/jenkins/working/art-tools_elliott_PR-525/.tox/py38/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)

  File "/mnt/workspace/jenkins/working/art-tools_elliott_PR-525/.tox/py38/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))

  File "/mnt/workspace/jenkins/working/art-tools_elliott_PR-525/.tox/py38/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)

  File "/mnt/workspace/jenkins/working/art-tools_elliott_PR-525/.tox/py38/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)

  File "/mnt/workspace/jenkins/working/art-tools_elliott_PR-525/.tox/py38/lib/python3.8/site-packages/click/decorators.py", line 38, in new_func
    return f(get_current_context().obj, *args, **kwargs)

  File "/mnt/workspace/jenkins/working/art-tools_elliott_PR-525/elliottlib/cli/find_bugs_blocker_cli.py", line 59, in find_bugs_blocker_cli
    find_bugs_obj = FindBugsBlocker()

  File "/mnt/workspace/jenkins/working/art-tools_elliott_PR-525/elliottlib/cli/find_bugs_blocker_cli.py", line 14, in __init__
    super().__init__(

TypeError: __init__() missing 1 required positional argument: 'cve_only'

======================================================================
FAIL: test_find_bugs_qe (tests.test_find_bugs_qe_cli.FindBugsQETestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/workspace/jenkins/working/art-tools_elliott_PR-525/tests/test_find_bugs_qe_cli.py", line 40, in test_find_bugs_qe
    self.assertIn(search_string1, result.output)
AssertionError: 'Found 1 bugs: OCPBUGS-123' not found in ''

======================================================================
FAIL: test_verify_bugs_with_sweep_cli (tests.test_verify_attached_bugs_cli.VerifyAttachedBugs)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/workspace/jenkins/working/art-tools_elliott_PR-525/.tox/py38/lib/python3.8/site-packages/asynctest/case.py", line 297, in run
    self._run_test_method(testMethod)
  File "/mnt/workspace/jenkins/working/art-tools_elliott_PR-525/.tox/py38/lib/python3.8/site-packages/asynctest/case.py", line 352, in _run_test_method
    result = method()
  File "/usr/lib64/python3.8/unittest/mock.py", line 1325, in patched
    return func(*newargs, **newkeywargs)
  File "/mnt/workspace/jenkins/working/art-tools_elliott_PR-525/tests/test_verify_attached_bugs_cli.py", line 57, in test_verify_bugs_with_sweep_cli
    self.assertIn('Regression possible: ON_QA bug OCPBUGS-2 is a backport of bug OCPBUGS-3 which has status MODIFIED',
AssertionError: 'Regression possible: ON_QA bug OCPBUGS-2 is a backport of bug OCPBUGS-3 which has status MODIFIED' not found in ''

----------------------------------------------------------------------
Ran 168 tests in 0.798s

FAILED (failures=3, errors=1)
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7f0ed82852b0>
ERROR: InvocationError for command /mnt/workspace/jenkins/working/art-tools_elliott_PR-525/.tox/py38/bin/coverage run --branch --source elliottlib -m unittest discover -t . -s tests/ (exited with code 1)
___________________________________ summary ____________________________________
ERROR:   py38: commands failed
openshift-bot commented 1 year ago

Build #2

GLOB sdist-make: /mnt/workspace/jenkins/working/art-tools_elliott_PR-525/setup.py
py38 recreate: /mnt/workspace/jenkins/working/art-tools_elliott_PR-525/.tox/py38
py38 installdeps: -rrequirements-dev.txt, -rrequirements.txt
py38 inst: /mnt/workspace/jenkins/working/art-tools_elliott_PR-525/.tox/.tmp/package/1/rh-elliott-2.0.15.dev26+g75a12be.zip
py38 installed: aiodns==3.0.0,aiohttp==3.8.4,aiosignal==1.3.1,astroid==2.15.0,async-timeout==4.0.2,asynctest==0.13.0,attrs==22.2.0,Brotli==1.0.9,cachetools==5.3.0,cchardet==2.1.7,certifi==2022.12.7,cffi==1.15.1,chardet==5.1.0,charset-normalizer==3.1.0,click==8.1.3,colorama==0.4.6,coverage==7.2.1,decorator==5.1.1,defusedxml==0.7.1,dill==0.3.6,distlib==0.3.6,errata-tool==1.30.0,exceptiongroup==1.1.0,filelock==3.9.0,flake8==6.0.0,flexmock==0.11.3,frozenlist==1.3.3,future==0.18.3,gssapi==1.8.2,idna==3.4,iniconfig==2.0.0,isort==5.12.0,jira==3.4.1,jsonpath-rw==1.4.0,koji==1.32.0,lazy-object-proxy==1.9.0,mccabe==0.7.0,mock==5.0.1,multidict==6.0.4,mypy==1.1.1,mypy-extensions==1.0.0,oauthlib==3.2.2,packaging==23.0,platformdirs==3.1.0,pluggy==1.0.0,ply==3.11,pycares==4.3.0,pycodestyle==2.10.0,pycparser==2.21,pyflakes==3.0.1,pygit2==1.10.1,pylint==2.16.4,pyproject_api==1.5.0,pytest==7.2.2,python-bugzilla==3.2.0,python-dateutil==2.8.2,PyYAML==6.0,requests==2.28.2,requests-gssapi==1.2.3,requests-oauthlib==1.3.1,requests-toolbelt==0.10.1,rh-elliott @ file:///mnt/workspace/jenkins/working/art-tools_elliott_PR-525/.tox/.tmp/package/1/rh-elliott-2.0.15.dev26%2Bg75a12be.zip,ruamel.yaml==0.17.21,ruamel.yaml.clib==0.2.7,semver==2.13.0,setuptools-scm==7.1.0,six==1.16.0,tenacity==8.2.2,tomli==2.0.1,tomlkit==0.11.6,tox==4.4.6,typing_extensions==4.5.0,urllib3==1.26.14,virtualenv==20.20.0,wrapt==1.15.0,yarl==1.8.2
py38 run-test-pre: PYTHONHASHSEED='1128525756'
py38 run-test: commands[0] | coverage run --branch --source elliottlib -m unittest discover -t . -s tests/
.............................................Cannot find any corresponding flaw bugs for these trackers: [13]. These trackers do not have a valid whiteboard component value: [12].
..BZ 9 is ignored because its status was MODIFIED at the moment of sweep cutoff (2021-06-30 12:30:00), however its status changed back to ['ASSIGNED'] afterwards
.....................changed 123 from status1 to status2
.123 is already on status1
.changed 123 from status1 to status2
..changed 123 from status1 to status2
.123 is already on status1
.changed 123 from status1 to status2
.................................................Bug(s) ['OCPBUGS-4', 'OCPBUGS-6', 'OCPBUGS-5'] have CVE in description but do not have TrackerBug labels. Please investigate.
.............................................
----------------------------------------------------------------------
Ran 168 tests in 0.764s

OK
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7f881d30beb0>
py38 run-test: commands[1] | coverage report
Name                                              Stmts   Miss Branch BrPart  Cover
-----------------------------------------------------------------------------------
elliottlib/__init__.py                               12      7      2      1    43%
elliottlib/_version.py                                2      2      0      0     0%
elliottlib/assembly.py                               92      9     54      7    89%
elliottlib/assertion.py                              11      0      6      0   100%
elliottlib/brew.py                                  328    188    134      5    36%
elliottlib/bug/__init__.py                            0      0      0      0   100%
elliottlib/build_finder.py                          114     12     74     15    86%
elliottlib/bzutil.py                                804    329    334     26    55%
elliottlib/cincinnati.py                             21     21      4      0     0%
elliottlib/cli/__init__.py                            0      0      0      0   100%
elliottlib/cli/__main__.py                          270    270     75      0     0%
elliottlib/cli/add_metadata_cli.py                   28     28      4      0     0%
elliottlib/cli/advisory_commons_cli.py               90     90     38      0     0%
elliottlib/cli/advisory_drop_cli.py                  37     37     14      0     0%
elliottlib/cli/advisory_images_cli.py                14     14      2      0     0%
elliottlib/cli/advisory_impetus_cli.py               22     22      6      0     0%
elliottlib/cli/attach_bugs_cli.py                    37     37     18      0     0%
elliottlib/cli/attach_cve_flaws_cli.py              130     66     58      9    45%
elliottlib/cli/change_state_cli.py                   44     44     16      0     0%
elliottlib/cli/cli_opts.py                           15      0     17      0   100%
elliottlib/cli/common.py                             48     12      6      1    69%
elliottlib/cli/create_cli.py                         70     70     20      0     0%
elliottlib/cli/create_placeholder_cli.py             36     36     12      0     0%
elliottlib/cli/create_textonly_cli.py                51     51      8      0     0%
elliottlib/cli/find_bugs_blocker_cli.py              43      4     10      3    87%
elliottlib/cli/find_bugs_qe_cli.py                   36      4      6      0    90%
elliottlib/cli/find_bugs_sweep_cli.py               218     33    124     21    82%
elliottlib/cli/find_builds_cli.py                   268    198    179      2    19%
elliottlib/cli/get_golang_versions_cli.py            59     14     32      9    73%
elliottlib/cli/list_cli.py                           14     14      2      0     0%
elliottlib/cli/puddle_advisories_cli.py              34     34      8      0     0%
elliottlib/cli/remove_bugs_cli.py                    46      6     14      6    80%
elliottlib/cli/repair_bugs_cli.py                    61      5     26      6    87%
elliottlib/cli/rhcos_cli.py                          93     93     52      0     0%
elliottlib/cli/rpmdiff_cli.py                       102    102     34      0     0%
elliottlib/cli/tag_builds_cli.py                    128    128     62      0     0%
elliottlib/cli/tarball_sources_cli.py                83     83     26      0     0%
elliottlib/cli/validate_rhsa.py                      45     45     16      0     0%
elliottlib/cli/verify_attached_bugs_cli.py          257     91    167     20    57%
elliottlib/cli/verify_attached_operators_cli.py     118     85     55      0    19%
elliottlib/cli/verify_cvp_cli.py                    165    165     78      0     0%
elliottlib/constants.py                              52      0      0      0   100%
elliottlib/cvp.py                                   216    216    113      0     0%
elliottlib/dotconfig.py                              54     11     30      6    75%
elliottlib/errata.py                                323    207    142      4    33%
elliottlib/errata_async.py                          151     14     58      9    89%
elliottlib/exceptions.py                             12      0      0      0   100%
elliottlib/exectools.py                              84      7     24      3    89%
elliottlib/gitdata.py                               171    137     76      0    14%
elliottlib/imagecfg.py                               21      2      0      0    90%
elliottlib/logutil.py                                 9      1      2      1    82%
elliottlib/metadata.py                              154     43     76     18    67%
elliottlib/model.py                                 105     22     30      2    79%
elliottlib/openshiftclient.py                        34     34      8      0     0%
elliottlib/pushd.py                                  21      0      2      0   100%
elliottlib/resultsdb.py                              28     28      4      0     0%
elliottlib/rhcos.py                                  63     63     16      0     0%
elliottlib/rpm_utils.py                              30     14     16      5    46%
elliottlib/rpmcfg.py                                  5      2      0      0    60%
elliottlib/rpmdiff.py                                31      0      0      0   100%
elliottlib/runtime.py                               278    202    106      2    22%
elliottlib/tarball_sources.py                        90     61     34      0    30%
elliottlib/util.py                                  297    139    110     15    50%
-----------------------------------------------------------------------------------
TOTAL                                              6275   3652   2640    196    39%
py38 run-test: commands[2] | flake8
___________________________________ summary ____________________________________
  py38: commands succeeded
  congratulations :)
rioliu-rh commented 1 year ago

@joepvd @thiagoalessio could you help to review this PR, thanks

joepvd commented 1 year ago

/lgtm