search

vmware-tanzu / tanzu-framework

Tanzu Framework provides a set of building blocks to build atop of the Tanzu platform and leverages Carvel packaging and plugins to provide users with a much stronger, more integrated experience than the loose coupling and stand-alone commands of the previous generation of tools.
Apache License 2.0
196 stars 193 forks source link

Flaky test in the `unittests` job #4405

Open srm09 opened 1 year ago

srm09 commented 1 year ago

Bug description Seen this test fail multiple times when running the unit tests job

• Failure [13.309 seconds]
ClusterBootstrap Reconciler
/home/runner/work/tanzu-framework/tanzu-framework/addons/controllers/clusterbootstrap_controller_test.go:42
  when cluster is created from clusterBootstrapTemplate
  /home/runner/work/tanzu-framework/tanzu-framework/addons/controllers/clusterbootstrap_controller_test.go:781
    from a ClusterBootstrapTemplate
    /home/runner/work/tanzu-framework/tanzu-framework/addons/controllers/clusterbootstrap_controller_test.go:787
      should perform ClusterBootstrap reconciliation & block reconciliation if ClusterBootstrap gets paused [It]
      /home/runner/work/tanzu-framework/tanzu-framework/addons/controllers/clusterbootstrap_controller_test.go:788

      Expected
          <bool>: true
      not to be true
      /home/runner/work/tanzu-framework/tanzu-framework/addons/controllers/clusterbootstrap_controller_test.go:914

Affected product area (please put an X in all that apply)

Expected behavior

srm09 commented 1 year ago

An example for this one https://github.com/vmware-tanzu/tanzu-framework/actions/runs/4196558719/jobs/7277690816#step:10:2519

srm09 commented 1 year ago

Rewriting this assertion https://github.com/vmware-tanzu/tanzu-framework/blame/bd3036ea658d6b7b0fc394ed796c98274b56bdb4/addons/controllers/clusterbootstrap_controller_test.go#L908-L916 might be helpful

// Wait 10 seconds in case reconciliation happens
// time.Sleep(10 * time.Second) <========= Remove this

Eventually(func() bool {
    remoteSecret := &corev1.Secret{}
    if err := k8sClient.Get(ctx, client.ObjectKey{Namespace: constants.TKGSystemNS, Name: util.GenerateDataValueSecretName(clusterName, foobar1CarvelPackageRefName)}, remoteSecret); err != nil {
        return false
    }
    // values.yaml should not update
    if string(s.Data["values.yaml"]) == string(remoteSecret.Data["values.yaml"]) {
        return false
    }
    return true
}, waitTimeout, pollingInterval).Should(BeTrue())// Wait 10 seconds in case reconciliation happens

only to confirm if the above assertion is what the original intention was

Note No need for the timeout since Eventually will keep retrying until the assertion is met or the timeout occurs whichever is first.