Bring 'test_products_found_deployment_report' back to execution

[ruda]

Bring middle-ware inspection test back to execution.

Sample output:

E       AssertionError: Found 1 unexpected products!                                                                                                                                                                                           
E         Errors are listed below:                                                                                                                                                                                                             
E          Found ['jbossfuse'] but only expected to find                                                                                                                                                                                       
E         ['jbosseap', 'jbosswebserver'] on discovery-mw.
...
FAILED camayoc/tests/qpc/api/v1/reports/test_reports.py::test_products_found_deployment_report[testlab-middleware] - AssertionError: Found 1 unexpected products!

Sample configuration:

  - name: testlab-middleware
    sources:
      - testlab-middleware
    expected_data:
      discovery-mw:
        distribution:
          release: "Red Hat Enterprise Linux release 9.2"
          name: "Red Hat Enterprise Linux"
          version: "9.2"
        products:
          - name: "JBoss EAP"
            presence: "present"
          - name: "JBoss Web Server"
            presence: "present"
          - name: "JBoss Fuse"
            presence: "absent"
          - name: "JBoss BRMS"
            presence: "absent"

Related to JIRA: DISCOVERY-432 https://issues.redhat.com/browse/DISCOVERY-432

[codecov[bot]]

Codecov Report

Merging #454 (766ab52) into main (b264311) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #454   +/-   ##
=======================================
  Coverage   77.07%   77.07%           
=======================================
  Files           4        4           
  Lines         205      205           
=======================================
  Hits          158      158           
  Misses         47       47           

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

[mirekdlugosz]

Just so I understand correctly: this PR brings back middleware test, but in our test lab we have single machine that has middlewares installed AND additional echo in login shell. As a result, a scan running against this machine will report that jbossfuse is installed on machine, which fails the test (because it is not installed and we don't expect to see it). Is that correct?

If that is what is happening, then I'm afraid we can't merge that - it will cause all pipeline runs (nightly and upstream PRs) to fail.

As a immediate workaround we could keep the test skipped. Then locally we can remove skip marker and execute a test, but pipelines are not affected. Of course we should un-skip test in main as soon as it can pass on pipelines.

[ruda]

Just so I understand correctly: this PR brings back middleware test, but in our test lab we have single machine that has middlewares installed AND additional echo in login shell. As a result, a scan running against this machine will report that jbossfuse is installed on machine, which fails the test (because it is not installed and we don't expect to see it). Is that correct?

Yes, that's correct.

If that is what is happening, then I'm afraid we can't merge that - it will cause all pipeline runs (nightly and upstream PRs) to fail.

I can disable the echo message at the level of the user that I had set up for login. When I disable the echo message from the bashrc file, then the test passes.

camayoc/tests/qpc/api/v1/reports/test_reports.py::test_products_found_deployment_report[testlab-middleware] PASSED [100%]

[ruda]

Here is a "happy path" execution so far:

camayoc/tests/qpc/api/v1/reports/test_reports.py::test_OS_found_deployment_report[Sat6] PASSED                               [ 14%]
camayoc/tests/qpc/api/v1/reports/test_reports.py::test_report_content_consistency PASSED                                     [ 28%]
camayoc/tests/qpc/api/v1/reports/test_reports.py::test_products_found_deployment_report[testlab] PASSED                      [ 42%]
camayoc/tests/qpc/api/v1/reports/test_reports.py::test_products_found_deployment_report[testlab-middleware] PASSED           [ 57%]
camayoc/tests/qpc/api/v1/reports/test_reports.py::test_OS_found_deployment_report[testlab] PASSED                            [ 71%]
camayoc/tests/qpc/api/v1/reports/test_reports.py::test_OS_found_deployment_report[testlab-middleware] PASSED                 [ 85%]
camayoc/tests/qpc/api/v1/reports/test_reports.py::test_OS_found_deployment_report[VCenterOnly] PASSED                        [100%]

[mirekdlugosz]

I am OK with not having coverage for https://issues.redhat.com/browse/DISCOVERY-407 in our test lab. Disabling echo sounds like a good idea to me.

Thanks for including other suggestions, and sorry for late response. I think we can merge this.