Add Conformance Profiles Test Reporting System

[tssurya]

Sample Output for Conformance Reports:

=== NAME  TestConformanceProfiles
    conformance_profile_test.go:142: Conformance report:
        apiVersion: policy.networking.k8s.io/v1alpha1
        date: "2023-10-19T19:03:23+02:00"
        implementation:
          additionalInformation: "https://github.com/ovn-org/ovn-kubernetes/blob/e0d2ab8afd043a03c6e6b2935a0d773810dad0f7/test/conformance/network_policy_v2_test.go#L50"
          contact:
          - '@tssurya'
          organization: ovn-org
          project: ovn-kubernetes
          url: https://github.com/ovn-org/ovn-kubernetes
          version: 0.1.1
        kind: ConformanceReport
        networkPolicyV2APIVersion: v0.1.2
        profiles:
        - core:
            result: success
            statistics:
              Failed: 0
              Passed: 7
              Skipped: 0
            summary: ""
          name: AdminNetworkPolicy
        - core:
            result: success
            statistics:
              Failed: 0
              Passed: 7
              Skipped: 0
            summary: ""
          name: BaselineAdminNetworkPolicy

How to run tests in profiles mode?

go test  -v ./conformance -run TestConformanceProfiles -args --conformance-profiles=AdminNetworkPolicy,BaselineAdminNetworkPolicy --organization=ovn-org -project=ovn-kubernetes -url="https://github.com/ovn-org/ovn-kubernetes" -version=0.1.1 -contact=@tssurya -additional-info="blah"

Related to #137

[k8s-ci-robot]

Skipping CI for Draft Pull Request. If you want CI signal for your change, please convert it to an actual PR. You can still manually trigger a test run with /test all

[netlify[bot]]

✅ Deploy Preview for kubernetes-sigs-network-policy-api ready!

Name	Link
🔨 Latest commit	7e9c9b451a66a606a098071a68e502102aa0cb63
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-sigs-network-policy-api/deploys/653165f36e7380000864efc3
😎 Deploy Preview	https://deploy-preview-142--kubernetes-sigs-network-policy-api.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[tssurya]

/area conformance

[tssurya]

@Dyanngg : PTAL! I am super interested to see if you can pull these changes in, bring up an antrea cluster, point the KUBECONFIG to it and then run:

go test  -v ./conformance -run TestConformanceProfiles -args --conformance-profiles=AdminNetworkPolicy,BaselineAdminNetworkPolicy --organization=ovn-org -project=ovn-kubernetes -url="https://github.com/ovn-org/ovn-kubernetes" -version=0.1.1 -contact=@tssurya -additional-info="blah"

something like that how does your test report look?

[Dyanngg]

@Dyanngg : PTAL! I am super interested to see if you can pull these changes in, bring up an antrea cluster, point the KUBECONFIG to it and then run: go test -v ./conformance -run TestConformanceProfiles -args --conformance-profiles=AdminNetworkPolicy,BaselineAdminNetworkPolicy --organization=ovn-org -project=ovn-kubernetes -url="https://github.com/ovn-org/ovn-kubernetes" -version=0.1.1 -contact=@tssurya -additional-info="blah" something like that how does your test report look?

Pretty much the same results:

=== NAME  TestConformanceProfiles
    conformance_profile_test.go:142: Conformance report:
        apiVersion: policy.networking.k8s.io/v1alpha1
        date: "2023-10-19T13:52:40-07:00"
        implementation:
          additionalInformation: blah
          contact:
          - '@dyanngg'
          organization: antrea.io
          project: antrea
          url: https://github.com/antrea-io/antrea
          version: v1.13
        kind: ConformanceReport
        networkPolicyV2APIVersion: v0.1.2
        profiles:
        - core:
            result: success
            statistics:
              Failed: 0
              Passed: 7
              Skipped: 0
            summary: ""
          name: AdminNetworkPolicy
        - core:
            result: success
            statistics:
              Failed: 0
              Passed: 7
              Skipped: 0
            summary: ""
          name: BaselineAdminNetworkPolicy

Although tbh I feel like the addtionalInfo section is a bit unclear/unstructured at this point. We might want to take a look around and see if there's a standardized way where CI info can be passed, i.e. maybe a struct that supports mainstream CIs like github action/jenkins/argo etc.

[tssurya]

Although tbh I feel like the addtionalInfo section is a bit unclear/unstructured at this point. We might want to take a look around and see if there's a standardized way where CI info can be passed, i.e. maybe a struct that supports mainstream CIs like github action/jenkins/argo etc.

I understand and agree fully. Let me open an issue to track this for us to do better moving forward. String is too loose ended.

[tssurya]

Opened https://github.com/kubernetes-sigs/network-policy-api/issues/160 to fix additional info field

[astoycos]

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: astoycos, Dyanngg, tssurya

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/kubernetes-sigs/network-policy-api/blob/main/OWNERS)~~ [Dyanngg,astoycos] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

[astoycos]

/lgtm