search

kubernetes-sigs / network-policy-api

This repo addresses further work involving Kubernetes network security beyond the initial NetworkPolicy resource
Apache License 2.0
54 stars 29 forks source link

Change conformance test ready condition #125

Closed Dyanngg closed 1 year ago

Dyanngg commented 1 year ago

In the conformance test suite, currently pod ready status is used to determine whether tests should be commenced. However, in some testbeds, listing pods in a namespace when the a StatefulSet has just been created will return empty list, thus bypassing the pod ready test, causing subsequent tests to fail:

=== RUN   TestNetworkPolicyV2Conformance
    adminnetworkpolicy_test.go:43: Configuring environment for network policy API conformance tests
    adminnetworkpolicy_test.go:60: Starting the network policy conformance test suite
    suite.go:116: Test Setup: Applying base manifests
    apply.go:124: Creating network-policy-conformance-gryffindor Namespace
    apply.go:124: Creating network-policy-conformance-slytherin Namespace
    apply.go:124: Creating network-policy-conformance-hufflepuff Namespace
    apply.go:124: Creating network-policy-conformance-ravenclaw Namespace
    apply.go:124: Creating harry-potter StatefulSet
    apply.go:124: Creating draco-malfoy StatefulSet
    apply.go:124: Creating cedric-diggory StatefulSet
    apply.go:124: Creating luna-lovegood StatefulSet
    suite.go:119: Test Setup: Ensuring Namespaces and Pods from base manifests are ready
    helper.go:95: Namespaces and Pods in network-policy-conformance-gryffindor, network-policy-conformance-slytherin, network-policy-conformance-hufflepuff, network-policy-conformance-ravenclaw namespaces ready
=== RUN   TestNetworkPolicyV2Conformance/AdminNetworkPolicyEgressSCTP
    suite.go:172: Applying base/admin_network_policy/core-egress-sctp-rules.yaml
    apply.go:124: Creating egress-sctp AdminNetworkPolicy
=== RUN   TestNetworkPolicyV2Conformance/AdminNetworkPolicyEgressSCTP/Should_support_an_'allow-egress'_policy_for_SCTP_protocol;_ensure_rule_ordering_is_respected
  Jul 17 22:03:18.000: INFO: Running '/usr/local/bin/kubectl --namespace=network-policy-conformance-ravenclaw exec luna-lovegood-0 -- /agnhost connect --timeout=3s --protocol=sctp :9003'
  Jul 17 22:03:18.271: INFO: rc: 1
  Jul 17 22:03:18.271: INFO: FAILED Command was [/agnhost connect --timeout=3s --protocol=sctp :9003]
  Jul 17 22:03:18.271: INFO: FAILED Response was , expected connection to succeed from luna-lovegood-0 to , but instead it miserably failed: error running /usr/local/bin/kubectl --namespace=network-policy-conformance-ravenclaw exec luna-lovegood-0 -- /agnhost connect --timeout=3s --protocol=sctp :9003:

In the above log, the IP of pod luna-lovegood-0 is not retrieved so the connection command [/agnhost connect --timeout=3s --protocol=sctp :9003] undoubtedly failed. This PR changes the readiness check to use the StatefulSet replica count in its status.

netlify[bot] commented 1 year ago

Deploy Preview for kubernetes-sigs-network-policy-api ready!

Name Link
Latest commit fb53f869f843a931d2070e64272373feb32b7f5d
Latest deploy log https://app.netlify.com/sites/kubernetes-sigs-network-policy-api/deploys/64b6cd5034d41100089e3836
Deploy Preview https://deploy-preview-125--kubernetes-sigs-network-policy-api.netlify.app
Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

tssurya commented 1 year ago

/test pull-network-policy-api-verify

tssurya commented 1 year ago

oh nice catch @Dyanngg : Actually I ran into the same issue:

2023-07-09T17:40:41.1665503Z === RUN   TestNetworkPolicyV2Conformance
2023-07-09T17:40:41.1669905Z     network_policy_v2_test.go:29: Configuring environment for network policy V2 API conformance tests
2023-07-09T17:40:41.2145112Z     network_policy_v2_test.go:46: Starting the network policy V2 conformance test suite
2023-07-09T17:40:41.2149069Z     suite.go:116: Test Setup: Applying base manifests
2023-07-09T17:40:41.4061275Z     apply.go:124: Creating network-policy-conformance-gryffindor Namespace
2023-07-09T17:40:41.4402524Z     apply.go:124: Creating network-policy-conformance-slytherin Namespace
2023-07-09T17:40:41.4782384Z     apply.go:124: Creating network-policy-conformance-hufflepuff Namespace
2023-07-09T17:40:41.5528767Z     apply.go:124: Creating network-policy-conformance-ravenclaw Namespace
2023-07-09T17:40:41.6029283Z     apply.go:124: Creating harry-potter StatefulSet
2023-07-09T17:40:41.6246020Z     apply.go:124: Creating draco-malfoy StatefulSet
2023-07-09T17:40:41.6389760Z     apply.go:124: Creating cedric-diggory StatefulSet
2023-07-09T17:40:41.6659390Z     apply.go:124: Creating luna-lovegood StatefulSet
2023-07-09T17:40:41.6781714Z     suite.go:119: Test Setup: Ensuring Namespaces and Pods from base manifests are ready
2023-07-09T17:40:41.7056381Z     helper.go:95: Namespaces and Pods in network-policy-conformance-gryffindor, network-policy-conformance-slytherin, network-policy-conformance-hufflepuff, network-policy-conformance-ravenclaw namespaces ready
2023-07-09T17:40:41.7057772Z === RUN   TestNetworkPolicyV2Conformance/AdminNetworkPolicyEgressSCTP
2023-07-09T17:40:41.7058571Z     suite.go:172: Applying base/admin_network_policy/core-egress-sctp-rules.yaml
2023-07-09T17:40:42.0461103Z     apply.go:124: Creating egress-sctp AdminNetworkPolicy
2023-07-09T17:40:42.0523483Z === RUN   TestNetworkPolicyV2Conformance/AdminNetworkPolicyEgressSCTP/Should_support_an_'allow-egress'_policy_for_SCTP_protocol;_ensure_rule_ordering_is_respected
2023-07-09T17:40:42.1625627Z   Jul  9 17:40:42.161: INFO: Running '/usr/local/bin/kubectl --namespace=network-policy-conformance-ravenclaw exec luna-lovegood-0 -- /agnhost connect --timeout=3s --protocol=sctp :9003'
2023-07-09T17:40:43.1633047Z   Jul  9 17:40:43.163: INFO: rc: 1
2023-07-09T17:40:43.1641751Z   Jul  9 17:40:43.163: INFO: FAILED Command was [/agnhost connect --timeout=3s --protocol=sctp :9003]
2023-07-09T17:40:43.1643518Z   Jul  9 17:40:43.163: INFO: FAILED Response was , expected connection to succeed from luna-lovegood-0 to , but instead it miserably failed: error running /usr/local/bin/kubectl --namespace=network-policy-conformance-ravenclaw exec luna-lovegood-0 -- /agnhost connect --timeout=3s --protocol=sctp :9003:
2023-07-09T17:40:43.1644338Z   Command stdout:

https://github.com/ovn-org/ovn-kubernetes/actions/runs/5500756505/jobs/10024064134 in my CI and was wondering why this happened. HAHA thanks for investigating and fixing this. /assign @tssurya

k8s-ci-robot commented 1 year ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Dyanngg, tssurya

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/kubernetes-sigs/network-policy-api/blob/master/OWNERS)~~ [Dyanngg] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
tssurya commented 1 year ago

/lgtm