search

submariner-io / subctl

CLI tool that provides helper commands to install, inspect, and troubleshoot a Submariner deployment.
Apache License 2.0
19 stars 23 forks source link

Make subctl verify output more useful #1099

Closed tpantelis closed 6 months ago

tpantelis commented 7 months ago

It's useful to see the names of test specs that were run. In order to do this, we need to set the Verbose flag in the ginkgo ReporterConfig but this also outputs the steps produced via the ginkgo.By function. We can use the --verbose CLI flag to suppress the ginkgo.By output by using the override provided by the shipyard framework. We can also eliminate the output for SynchronizedBeforeSuite/SynchronizedAfterSuite by calling framework.BeforeSuite/framework.RunCleanupActions directly.

See individual commits for details.

Here's the condensed output without the --verbose flag:

Performing the following verifications: basic-connectivity
  Running Suite: Submariner E2E suite - /go/src/github.com/submariner-io/subctl
  =============================================================================
  Random Seed: 1

  Will run 4 of 47 specs
  SSSSSS
  ------------------------------
  Basic TCP connectivity tests across clusters without discovery when a pod connects via TCP to a remote pod when the pod is not on a gateway and the remote pod is not on a gateway should have sent the expected data from the pod to the other pod [dataplane, basic]
  github.com/submariner-io/submariner@v0.18.0-m2.0.20240403165334-104a258289ca/test/e2e/dataplane/tcp_pod_connectivity.go:38
  • [3.538 seconds]
  ------------------------------
  SS
  ------------------------------
  Basic TCP connectivity tests across clusters without discovery when a pod connects via TCP to a remote pod when the pod is on a gateway and the remote pod is on a gateway should have sent the expected data from the pod to the other pod [dataplane, basic]
  github.com/submariner-io/submariner@v0.18.0-m2.0.20240403165334-104a258289ca/test/e2e/dataplane/tcp_pod_connectivity.go:38
  • [3.537 seconds]
  ------------------------------
  Basic TCP connectivity tests across clusters without discovery when a pod connects via TCP to a remote service when the pod is not on a gateway and the remote service is not on a gateway should have sent the expected data from the pod to the other pod [dataplane, basic]
  github.com/submariner-io/submariner@v0.18.0-m2.0.20240403165334-104a258289ca/test/e2e/dataplane/tcp_pod_connectivity.go:38
  • [4.048 seconds]
  ------------------------------
  SS
  ------------------------------
  Basic TCP connectivity tests across clusters without discovery when a pod connects via TCP to a remote service when the pod is on a gateway and the remote service is on a gateway should have sent the expected data from the pod to the other pod [dataplane, basic]
  github.com/submariner-io/submariner@v0.18.0-m2.0.20240403165334-104a258289ca/test/e2e/dataplane/tcp_pod_connectivity.go:38
  • [4.550 seconds]
  ------------------------------
  SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS

  Ran 4 of 47 Specs in 15.676 seconds
  SUCCESS! -- 4 Passed | 0 Failed | 0 Pending | 43 Skipped

Here's the output with the --verbose flag (truncated to show only one test):

Performing the following verifications: basic-connectivity
  Apr  4 15:09:30.662: Creating kubernetes clients
  Apr  4 15:09:30.673: Creating submariner clients
  Apr  4 15:09:30.679: Creating lighthouse clients
  Running Suite: Submariner E2E suite - /go/src/github.com/submariner-io/subctl
  =============================================================================
  Random Seed: 1

  Will run 4 of 47 specs
  SSSSSS
  Basic TCP connectivity tests across clusters without discovery when a pod connects via TCP to a remote pod when the pod is not on a gateway and the remote pod is not on a gateway should have sent the expected data from the pod to the other pod [dataplane, basic]
  github.com/submariner-io/submariner@v0.18.0-m2.0.20240403165334-104a258289ca/test/e2e/dataplane/tcp_pod_connectivity.go:38
  Apr  4 15:09:30.709: Creating namespace objects with basename "dataplane-conn-nd"
  Apr  4 15:09:30.712: Generated namespace "e2e-tests-dataplane-conn-nd-gqmfj" in cluster "cluster1" to execute the tests in
  Apr  4 15:09:30.712: Creating namespace "e2e-tests-dataplane-conn-nd-gqmfj" in cluster "cluster2"
  Apr  4 15:09:30.718: Creating a listener pod in cluster "cluster2", which will wait for a handshake over TCP
  Apr  4 15:09:31.727: Apr  4 15:09:31.727: INFO: Will send traffic to IP: 10.131.0.12

  Apr  4 15:09:31.727: Creating a connector pod in cluster "cluster1", which will attempt the specific UUID handshake over TCP
  Apr  4 15:09:31.733: Waiting for the connector pod "tcp-check-podw8qq7" to exit, returning what connector sent
  Apr  4 15:09:34.737: Apr  4 15:09:34.737: INFO: Pod "tcp-check-podw8qq7" on node "cluster1-control-plane" output:
  10.131.0.12 (10.131.0.12:1234) open
  [dataplane] listener says a3fe9f37-78e2-4881-81f8-2004c1f9f2d3

  Apr  4 15:09:34.737: Waiting for the listener pod "tcp-check-listenernjcvv" to exit, returning what listener sent
  Apr  4 15:09:34.739: Apr  4 15:09:34.739: INFO: Pod "tcp-check-listenernjcvv" on node "cluster2-control-plane" output:
  listening on 0.0.0.0:1234 ...
  connect to 10.131.0.12:1234 from 10.130.0.16:45981 (10.130.0.16:45981)
  [dataplane] connector says c0908c2b-ff48-4138-ad45-9f44ee9d1c65

  Apr  4 15:09:34.739: Apr  4 15:09:34.739: INFO: Connector pod has IP: 

  Apr  4 15:09:34.739: Verifying that the listener got the connector's data and the connector got the listener's data
  Apr  4 15:09:34.739: Verifying the output of listener pod which must contain the source IP
  Apr  4 15:09:34.739: Deleting namespace "e2e-tests-dataplane-conn-nd-gqmfj" on cluster "cluster1"
  Apr  4 15:09:34.743: Deleting namespace "e2e-tests-dataplane-conn-nd-gqmfj" on cluster "cluster2"
  • [4.037 seconds]
  ------------------------------
  <snip>
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS

  Ran 4 of 47 Specs in 16.195 seconds
  SUCCESS! -- 4 Passed | 0 Failed | 0 Pending | 43 Skipped

Fixes https://github.com/submariner-io/subctl/issues/131

submariner-bot commented 7 months ago

🤖 Created branch: z_pr1099/tpantelis/verify_output 🚀 Full E2E won't run until the "ready-to-test" label is applied. I will add it automatically once the PR has 2 approvals, or you can add it manually.

submariner-bot commented 6 months ago

🤖 Closed branches: [z_pr1099/tpantelis/verify_output]