Generic comparison

[paigerube14]

Type of change

• [x] Refactor
• [x] New feature
• [ ] Bug fix
• [ ] Optimization
• [ ] Documentation Update

Description

Using the changes in fmatch pr I was able to add in more options into the yaml and have a working ingress perf comparison

Related Tickets & Documents

• Related Issue #
• Closes #

Checklist before requesting a review

• [X] I have performed a self-review of my code.
• [ ] If it is a core feature, I have added thorough tests.

Testing

• Please describe the System Under Test.
• Please provide detailed steps to perform tests related to this code change.
• How were the fix/results from this change verified? Please provide relevant screenshots or results.

2 sample configs:

tests :
  - name : rosa-small-scale-ingress
    platform: AWS
    workerNodesType: m5.xlarge
    workerNodesCount: 9
    benchmark: ingress-perf
    ocpVersion: 4.15
    networkType: OVNKubernetes
    clusterType: rosa
    masterNodesCount: 3
    # encrypted: true
    # fips: false
    # ipsec: false
    #    masterNodesType: m6a.xlarge
    #    masterNodesCount: 3

    metrics : 
    - metric: Termination
      config.termination: passthrough
      sample: "*"
      config.serverReplicas: "*"
      metric_of_interest: total_avg_rps

tests :
  - name : rosa-small-scale-cluster-density-v2
    platform: AWS
    workerNodesType: m5.xlarge
    workerNodesCount: 24
    benchmark: cluster-density-v2
    ocpVersion: 4.15
    networkType: OVNKubernetes
    clusterType: rosa
    masterNodesCount: 3
    # encrypted: true
    # fips: false
    # ipsec: false
    #    masterNodesType: m6a.xlarge
    #    masterNodesCount: 3

    metrics : 
    - metric: podReadyLatency
      metricName: podLatencyQuantilesMeasurement
      quantileName: Ready
      metric_of_interest: P99
      not: 
      - jobConfig.name: "garbage-collection"

    - metric : apiserverCPU
      metricName : containerCPU
      labels.namespace: openshift-kube-apiserver
      metric_of_interest: value
      agg:
        value: cpu
        agg_type: avg

    - metric : ovnCPU
      metricName : containerCPU
      labels.namespace: openshift-ovn-kubernetes
      metric_of_interest: value
      agg:
        value: cpu
        agg_type: avg

    - metric : etcdCPU
      metricName : containerCPU
      labels.namespace: openshift-etcd
      metric_of_interest: value
      agg:
        value: cpu
        agg_type: avg

    - metric : etcdDisck
      metricName : 99thEtcdDiskBackendCommitDurationSeconds
      metric_of_interest: value
      agg:
        value: duration
        agg_type: avg

[paigerube14]

This does include changes from https://github.com/cloud-bulldozer/orion/pull/7 as well

[shashank-boyapally]

lgtm!! thank you!

[paigerube14]

I think we'll need a new release of py-commons to incorporate these changes: https://github.com/cloud-bulldozer/py-commons/pull/16

[shashank-boyapally]

I'll cut a new release!