DISCOVERY-281 - Add test coverage for OCP Clusters and Nodes

[ruda]

Fix: DISCOVERY-281.

An OpenShift cluster is one or more instances belonging to a single product installation. An instance is a host or VM running in the cloud. Each cluster will have a unique name called the cluster_id.

The objective of this test is to verify if Quipucords is performing OpenShift (OCP) inspection and collecting correct facts at the level of clusters and nodes.

Camayoc config.yaml for openshift with cluster and nodes.

openshift:
  - hostname: api.sharedocp4upi412ovn.example.com
    port: 6443
    token: 'sha256~...'
    skip_tls_verify: true
    cluster_id: '97c44acf-e8f1-46eb-b6a5-6ed78ecf2659'
    version: '4.12.5'
    nodes: 6
    hosts:
      - 'master-0.sharedocp4upi412ovn.example.com'
      - 'master-1.sharedocp4upi412ovn.example.com'
      - 'master-2.sharedocp4upi412ovn.example.com'
      - 'worker-0.sharedocp4upi412ovn.example.com'
      - 'worker-1.sharedocp4upi412ovn.example.com'
      - 'worker-2.sharedocp4upi412ovn.example.com'

Open to discussion:

• What is the best place for the openshift configuration?
• Should split this in credential and source parts?
• Cluster provisioning and how to skip this test.
• How to handle token expiration?

[mirekdlugosz]

What is the best place for the openshift configuration?

There are still few things I wanted to do before I open it up for review, but I am working on this in config-refactor branch.

In general, I assume config file should follow general application data flow. So things like hostname, token and ssl cert verification should be part of credential. Things like cluster_id, version and hosts should be part of scan. In my PR, report data belongs to a scan, because that is how you obtain it. But I guess we can introduce top-level field reports that would reference scans.

We can discuss best approach here.

I am fine with merging config as you have it, and then changing openshift options in my PR.

Cluster provisioning and how to skip this test.

I feel provisioning is much more general problem, and I would put it out of scope of this PR. I think test introduced here should be able to assume that cluster exists and compare it to data specified in config file.

As we discussed previously, personally I think that infrastructure management does not belong to camayoc-the-test-runner, i.e. whatever happens when you run pytest inside camayoc source directory. Any automation we might have might belong to camayoc and we might keep it in this repository, but there should be different entry point.

How to handle token expiration?

dynaconf allows you to set any setting through environment variable. In Jenkins, we might introduce field to set custom environment variables, so you can pass it on for test run.

For recurring Jenkins jobs, we should explore long-living OpenShift API tokens, or have some automation that is able to obtain this token and pass it on. Such automation could be separate pipeline step, or implemented in camayoc itself.

Secret management is one of the topics we have discussed and while I can't find a Jira ticket right now, I'm sure it is there somewhere. Again - I would consider this out of scope of this PR and for now live with test passing only in standalone run.

[codecov[bot]]

Codecov Report

Merging #389 (a9ce7dd) into master (b2535a2) will not change coverage. The diff coverage is n/a.

:exclamation: Current head a9ce7dd differs from pull request most recent head 673d87c. Consider uploading reports for the commit 673d87c to get more accurate results

@@           Coverage Diff           @@
##           master     #389   +/-   ##
=======================================
  Coverage   80.57%   80.57%           
=======================================
  Files           5        5           
  Lines         242      242           
=======================================
  Hits          195      195           
  Misses         47       47           

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

[ruda]

@mirekdlugosz this test will be skipped if there isn't any clusters configured for testing, it's safe to be included in the pipeline.

pytest -v -ra camayoc/tests/qpc/cli
...
SKIPPED [1] camayoc/tests/qpc/cli/test_openshift.py:82: got empty parameter set ['cluster'], function test_openshift_clusters at /Users/rmoura/Developer/camayoc/camayoc/tests/qpc/cli/test_openshift.py:81
...

[mirekdlugosz]

As discussed yesterday, I'm merging this. It won't run on Jenkins currently, so it's safe to merge, and we can fix remaining issues later.

Thanks @ruda !