search

vmware-tanzu / community-edition

VMware Tanzu Community Edition is no longer an actively maintained project. Code is available for historical purposes only.
https://tanzucommunityedition.io/
Apache License 2.0
1.34k stars 308 forks source link

Cert-manager E2E-test is failing #4170

Closed aman556 closed 2 years ago

aman556 commented 2 years ago

Possible reason behind the error

Files with -values.yaml got ignored

Bug Report

On running E2E test for cert-manager it is failing with the error

make e2e-test
CGO_ENABLED=0 go run github.com/onsi/ginkgo/ginkgo -v e2e
Running Suite: cert-manager Package E2E Test Suite
==================================================
Random Seed: 1650888896
Will run 1 of 1 specs

STEP: installing cert-manager package
+ tanzu package available list -o json
[
  {
    "display-name": "App-Toolkit package for TCE",
    "latest-version": "0.1.0",
    "name": "app-toolkit.community.tanzu.vmware.com",
    "short-description": "Kubernetes-native toolkit to support application lifecycle"
  },
  {
    "display-name": "Cartographer",
    "latest-version": "0.3.0",
    "name": "cartographer.community.tanzu.vmware.com",
    "short-description": "Kubernetes native Supply Chain Choreographer."
  },
  {
    "display-name": "cert-injection-webhook",
    "latest-version": "0.1.1",
    "name": "cert-injection-webhook.community.tanzu.vmware.com",
    "short-description": "The Cert Injection Webhook injects CA certificates and proxy environment variables into pods"
  },
  {
    "display-name": "cert-manager",
    "latest-version": "1.8.0",
    "name": "cert-manager.community.tanzu.vmware.com",
    "short-description": "Certificate management"
  },
  {
    "display-name": "contour",
    "latest-version": "1.20.1",
    "name": "contour.community.tanzu.vmware.com",
    "short-description": "An ingress controller"
  },
  {
    "display-name": "external-dns",
    "latest-version": "0.10.0",
    "name": "external-dns.community.tanzu.vmware.com",
    "short-description": "This package provides DNS synchronization functionality."
  },
  {
    "display-name": "fluent-bit",
    "latest-version": "1.7.5",
    "name": "fluent-bit.community.tanzu.vmware.com",
    "short-description": "Fluent Bit is a fast Log Processor and Forwarder"
  },
  {
    "display-name": "Flux Source Controller",
    "latest-version": "0.21.2",
    "name": "fluxcd-source-controller.community.tanzu.vmware.com",
    "short-description": "The source-controller is a Kubernetes operator, specialised in artifacts acquisition from external sources such as Git, Helm repositories and S3 buckets."
  },
  {
    "display-name": "gatekeeper",
    "latest-version": "3.7.1",
    "name": "gatekeeper.community.tanzu.vmware.com",
    "short-description": "policy management"
  },
  {
    "display-name": "grafana",
    "latest-version": "7.5.11",
    "name": "grafana.community.tanzu.vmware.com",
    "short-description": "Visualization and analytics software"
  },
  {
    "display-name": "harbor",
    "latest-version": "2.4.2",
    "name": "harbor.community.tanzu.vmware.com",
    "short-description": "OCI Registry"
  },
  {
    "display-name": "knative-serving",
    "latest-version": "1.0.0",
    "name": "knative-serving.community.tanzu.vmware.com",
    "short-description": "Knative Serving builds on Kubernetes to support deploying and serving of applications and functions as serverless containers"
  },
  {
    "display-name": "kpack dependencies",
    "latest-version": "0.0.9",
    "name": "kpack-dependencies.community.tanzu.vmware.com",
    "short-description": "Dependencies in the form of Buildpacks and Stacks for the kpack package"
  },
  {
    "display-name": "kpack",
    "latest-version": "0.5.2",
    "name": "kpack.community.tanzu.vmware.com",
    "short-description": "kpack builds application source code into OCI compliant images using Cloud Native Buildpacks"
  },
  {
    "display-name": "Flux Kustomize Controller",
    "latest-version": "0.21.1",
    "name": "kustomize-controller.fluxcd.community.tanzu.vmware.com",
    "short-description": "Kustomize controller is one of the components in GitOps toolkit."
  },
  {
    "display-name": "local-path-storage",
    "latest-version": "0.0.22",
    "name": "local-path-storage.community.tanzu.vmware.com",
    "short-description": "This package provides local path node storage and primarily supports RWO AccessMode."
  },
  {
    "display-name": "multus-cni",
    "latest-version": "3.8.0",
    "name": "multus-cni.community.tanzu.vmware.com",
    "short-description": "This package provides the ability for enabling attaching multiple network interfaces to pods in Kubernetes"
  },
  {
    "display-name": "prometheus",
    "latest-version": "2.27.0-1",
    "name": "prometheus.community.tanzu.vmware.com",
    "short-description": "A time series database for your metrics"
  },
  {
    "display-name": "velero",
    "latest-version": "1.8.0",
    "name": "velero.community.tanzu.vmware.com",
    "short-description": "Disaster recovery capabilities"
  },
  {
    "display-name": "whereabouts",
    "latest-version": "0.5.1",
    "name": "whereabouts.community.tanzu.vmware.com",
    "short-description": "A CNI IPAM plugin that assigns IP addresses cluster-wide"
  }
]
+ tanzu package available list cert-manager.community.tanzu.vmware.com -o json
[
  {
    "name": "cert-manager.community.tanzu.vmware.com",
    "released-at": "2021-08-23 22:52:51 +0530 IST",
    "version": "1.5.4"
  },
  {
    "name": "cert-manager.community.tanzu.vmware.com",
    "released-at": "2021-08-23 22:52:51 +0530 IST",
    "version": "1.5.5"
  },
  {
    "name": "cert-manager.community.tanzu.vmware.com",
    "released-at": "2021-10-29 17:30:00 +0530 IST",
    "version": "1.6.1"
  },
  {
    "name": "cert-manager.community.tanzu.vmware.com",
    "released-at": "2021-10-29 17:30:00 +0530 IST",
    "version": "1.6.3"
  },
  {
    "name": "cert-manager.community.tanzu.vmware.com",
    "released-at": "2021-10-29 17:30:00 +0530 IST",
    "version": "1.7.2"
  },
  {
    "name": "cert-manager.community.tanzu.vmware.com",
    "released-at": "2021-10-29 17:30:00 +0530 IST",
    "version": "1.8.0"
  }
]
+ tanzu package install cert-manager --poll-interval 10s --poll-timeout 20m --namespace default --package-name cert-manager.community.tanzu.vmware.com --version 1.8.0 --values-file fixtures/cert-manager-values.yaml
Error: failed to read from data values file 'fixtures/cert-manager-values.yaml': open fixtures/cert-manager-values.yaml: no such file or directory
Error: exit status 1

Failure [4.149 seconds]
[BeforeSuite] BeforeSuite 
/Users/amansha/go/src/github.com/vmware-tanzu/community-edition/addons/packages/cert-manager/1.8.0/test/e2e/cert_manager_suite_test.go:47

  Unexpected error:
      <*errors.errorString | 0xc00005a0c0>: {
          s: "Error: failed to read from data values file 'fixtures/cert-manager-values.yaml': open fixtures/cert-manager-values.yaml: no such file or directory\nError: exit status 1\n\nexit status: 1",
      }
      Error: failed to read from data values file 'fixtures/cert-manager-values.yaml': open fixtures/cert-manager-values.yaml: no such file or directory
      Error: exit status 1

      exit status: 1
  occurred

  /Users/amansha/go/src/github.com/vmware-tanzu/community-edition/addons/packages/cert-manager/1.8.0/test/e2e/cert_manager_suite_test.go:131
------------------------------
STEP: cleaning up cert-manager package
+ tanzu package installed delete cert-manager --poll-interval 10s --poll-timeout 20m --namespace default --yes

STEP: validating the cert-manager package install no longer exists
+ kubectl -n default get packageinstall cert-manager
Error from server (NotFound): packageinstalls.packaging.carvel.dev "cert-manager" not found

STEP: cleaning up cert-manager namespace
+ kubectl delete namespace cert-manager
Error from server (NotFound): namespaces "cert-manager" not found

Ran 1 of 0 Specs in 4.919 seconds
FAIL! -- 0 Passed | 1 Failed | 0 Pending | 0 Skipped
--- FAIL: Test (4.92s)
FAIL

Ginkgo ran 1 suite in 6.656836347s
Test Suite Failed
exit status 1
make: *** [e2e-test] Error 1

Expected Behavior

Steps to Reproduce the Bug

cd addons/packages/cert-manager/1.8.0/test && make e2e-test

Screenshots or additional information and context

Environment Details

Diagnostics and log bundle

joshrosso commented 2 years ago

Thanks @aman556. Let's have @seemiller take a look at this ASAP.

joshrosso commented 2 years ago

Resolved via https://github.com/vmware-tanzu/community-edition/pull/4174#pullrequestreview-952062109.

@aman556 can you re-test and let us know if the issue resolves?

aman556 commented 2 years ago

Hey @joshrosso and @seemiller it’s working now. Thanks!

aman556 commented 2 years ago

Pass logs

make e2e-test
CGO_ENABLED=0 go run github.com/onsi/ginkgo/ginkgo -v e2e
Running Suite: cert-manager Package E2E Test Suite
==================================================
Random Seed: 1650901782
Will run 1 of 1 specs

STEP: installing cert-manager package
+ tanzu package available list -o json
[
  {
    "display-name": "App-Toolkit package for TCE",
    "latest-version": "0.1.0",
    "name": "app-toolkit.community.tanzu.vmware.com",
    "short-description": "Kubernetes-native toolkit to support application lifecycle"
  },
  {
    "display-name": "Cartographer",
    "latest-version": "0.3.0",
    "name": "cartographer.community.tanzu.vmware.com",
    "short-description": "Kubernetes native Supply Chain Choreographer."
  },
  {
    "display-name": "cert-injection-webhook",
    "latest-version": "0.1.1",
    "name": "cert-injection-webhook.community.tanzu.vmware.com",
    "short-description": "The Cert Injection Webhook injects CA certificates and proxy environment variables into pods"
  },
  {
    "display-name": "cert-manager",
    "latest-version": "1.8.0",
    "name": "cert-manager.community.tanzu.vmware.com",
    "short-description": "Certificate management"
  },
  {
    "display-name": "contour",
    "latest-version": "1.20.1",
    "name": "contour.community.tanzu.vmware.com",
    "short-description": "An ingress controller"
  },
  {
    "display-name": "external-dns",
    "latest-version": "0.10.0",
    "name": "external-dns.community.tanzu.vmware.com",
    "short-description": "This package provides DNS synchronization functionality."
  },
  {
    "display-name": "fluent-bit",
    "latest-version": "1.7.5",
    "name": "fluent-bit.community.tanzu.vmware.com",
    "short-description": "Fluent Bit is a fast Log Processor and Forwarder"
  },
  {
    "display-name": "Flux Source Controller",
    "latest-version": "0.21.2",
    "name": "fluxcd-source-controller.community.tanzu.vmware.com",
    "short-description": "The source-controller is a Kubernetes operator, specialised in artifacts acquisition from external sources such as Git, Helm repositories and S3 buckets."
  },
  {
    "display-name": "gatekeeper",
    "latest-version": "3.7.1",
    "name": "gatekeeper.community.tanzu.vmware.com",
    "short-description": "policy management"
  },
  {
    "display-name": "grafana",
    "latest-version": "7.5.11",
    "name": "grafana.community.tanzu.vmware.com",
    "short-description": "Visualization and analytics software"
  },
  {
    "display-name": "harbor",
    "latest-version": "2.4.2",
    "name": "harbor.community.tanzu.vmware.com",
    "short-description": "OCI Registry"
  },
  {
    "display-name": "knative-serving",
    "latest-version": "1.0.0",
    "name": "knative-serving.community.tanzu.vmware.com",
    "short-description": "Knative Serving builds on Kubernetes to support deploying and serving of applications and functions as serverless containers"
  },
  {
    "display-name": "kpack dependencies",
    "latest-version": "0.0.9",
    "name": "kpack-dependencies.community.tanzu.vmware.com",
    "short-description": "Dependencies in the form of Buildpacks and Stacks for the kpack package"
  },
  {
    "display-name": "kpack",
    "latest-version": "0.5.2",
    "name": "kpack.community.tanzu.vmware.com",
    "short-description": "kpack builds application source code into OCI compliant images using Cloud Native Buildpacks"
  },
  {
    "display-name": "Flux Kustomize Controller",
    "latest-version": "0.21.1",
    "name": "kustomize-controller.fluxcd.community.tanzu.vmware.com",
    "short-description": "Kustomize controller is one of the components in GitOps toolkit."
  },
  {
    "display-name": "local-path-storage",
    "latest-version": "0.0.22",
    "name": "local-path-storage.community.tanzu.vmware.com",
    "short-description": "This package provides local path node storage and primarily supports RWO AccessMode."
  },
  {
    "display-name": "multus-cni",
    "latest-version": "3.8.0",
    "name": "multus-cni.community.tanzu.vmware.com",
    "short-description": "This package provides the ability for enabling attaching multiple network interfaces to pods in Kubernetes"
  },
  {
    "display-name": "prometheus",
    "latest-version": "2.27.0-1",
    "name": "prometheus.community.tanzu.vmware.com",
    "short-description": "A time series database for your metrics"
  },
  {
    "display-name": "velero",
    "latest-version": "1.8.0",
    "name": "velero.community.tanzu.vmware.com",
    "short-description": "Disaster recovery capabilities"
  },
  {
    "display-name": "whereabouts",
    "latest-version": "0.5.1",
    "name": "whereabouts.community.tanzu.vmware.com",
    "short-description": "A CNI IPAM plugin that assigns IP addresses cluster-wide"
  }
]
+ tanzu package available list cert-manager.community.tanzu.vmware.com -o json
[
  {
    "name": "cert-manager.community.tanzu.vmware.com",
    "released-at": "2021-08-23 22:52:51 +0530 IST",
    "version": "1.5.4"
  },
  {
    "name": "cert-manager.community.tanzu.vmware.com",
    "released-at": "2021-08-23 22:52:51 +0530 IST",
    "version": "1.5.5"
  },
  {
    "name": "cert-manager.community.tanzu.vmware.com",
    "released-at": "2021-10-29 17:30:00 +0530 IST",
    "version": "1.6.1"
  },
  {
    "name": "cert-manager.community.tanzu.vmware.com",
    "released-at": "2021-10-29 17:30:00 +0530 IST",
    "version": "1.6.3"
  },
  {
    "name": "cert-manager.community.tanzu.vmware.com",
    "released-at": "2021-10-29 17:30:00 +0530 IST",
    "version": "1.7.2"
  },
  {
    "name": "cert-manager.community.tanzu.vmware.com",
    "released-at": "2021-10-29 17:30:00 +0530 IST",
    "version": "1.8.0"
  }
]
+ tanzu package install cert-manager --poll-interval 10s --poll-timeout 20m --namespace default --package-name cert-manager.community.tanzu.vmware.com --version 1.8.0 --values-file fixtures/values.yaml

STEP: validating cert-manager package is reconciled
+ kubectl -n default get packageinstalls cert-manager -o jsonpath={.status.conditions[?(@.type == 'ReconcileSucceeded')].status}
True
STEP: wait for cert-manager to be ready
+ kubectl get validatingwebhookconfigurations.admissionregistration.k8s.io cert-manager-webhook -o jsonpath={.webhooks[].clientConfig.caBundle}
LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJ3akNDQVVpZ0F3SUJBZ0lSQUxkMkJwbkw2VU1ZRWJsSFc5Znh3aUl3Q2dZSUtvWkl6ajBFQXdNd0lqRWcKTUI0R0ExVUVBeE1YWTJWeWRDMXRZVzVoWjJWeUxYZGxZbWh2YjJzdFkyRXdIaGNOTWpJd05ESTFNVFUxTURBMwpXaGNOTWpNd05ESTFNVFUxTURBM1dqQWlNU0F3SGdZRFZRUURFeGRqWlhKMExXMWhibUZuWlhJdGQyVmlhRzl2CmF5MWpZVEIyTUJBR0J5cUdTTTQ5QWdFR0JTdUJCQUFpQTJJQUJGZld5NCtVU0hiTFdUcG82Q1RPVWg2VExjYVUKbjMyM0UxOUs2QkRtSTJqSXd0OXJ6a1R6SVpFYi9ocWtlakc4bDE5WllBUndrQTJYWDFPZU8rOVBoSjBSeUowRwo2OWdLb2lzUnFnOXFsVDF1dFRKQ1MxZThPT29RU1lDeFRNV2FRcU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trCk1BOEdBMVVkRXdFQi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZPaFVsR1VyblpxdVhiaEJpQ1hSL1hlUVBTaXMKTUFvR0NDcUdTTTQ5QkFNREEyZ0FNR1VDTVFDYm5IdWlqa3B6RU01RWtHaTJSc1d5bXY4bGJLYmR3QklWRlB5bApUQ054KzB1RHFLOENUT1hiYXAwVjdqaWYvM3NDTUEyZjhpRlducUI5ZU9vVGNXajFSbzFVd0pWbG9kbnBUczNVClk4V0xpYm9QMUdtRm5VYVVtR3g0b05xOEI3Znhydz09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJ3akNDQVVpZ0F3SUJBZ0lSQUxkMkJwbkw2VU1ZRWJsSFc5Znh3aUl3Q2dZSUtvWkl6ajBFQXdNd0lqRWcKTUI0R0ExVUVBeE1YWTJWeWRDMXRZVzVoWjJWeUxYZGxZbWh2YjJzdFkyRXdIaGNOTWpJd05ESTFNVFUxTURBMwpXaGNOTWpNd05ESTFNVFUxTURBM1dqQWlNU0F3SGdZRFZRUURFeGRqWlhKMExXMWhibUZuWlhJdGQyVmlhRzl2CmF5MWpZVEIyTUJBR0J5cUdTTTQ5QWdFR0JTdUJCQUFpQTJJQUJGZld5NCtVU0hiTFdUcG82Q1RPVWg2VExjYVUKbjMyM0UxOUs2QkRtSTJqSXd0OXJ6a1R6SVpFYi9ocWtlakc4bDE5WllBUndrQTJYWDFPZU8rOVBoSjBSeUowRwo2OWdLb2lzUnFnOXFsVDF1dFRKQ1MxZThPT29RU1lDeFRNV2FRcU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trCk1BOEdBMVVkRXdFQi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZPaFVsR1VyblpxdVhiaEJpQ1hSL1hlUVBTaXMKTUFvR0NDcUdTTTQ5QkFNREEyZ0FNR1VDTVFDYm5IdWlqa3B6RU01RWtHaTJSc1d5bXY4bGJLYmR3QklWRlB5bApUQ054KzB1RHFLOENUT1hiYXAwVjdqaWYvM3NDTUEyZjhpRlducUI5ZU9vVGNXajFSbzFVd0pWbG9kbnBUczNVClk4V0xpYm9QMUdtRm5VYVVtR3g0b05xOEI3Znhydz09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
cert-manager Package E2E Test 
  works
  /Users/amansha/go/src/github.com/vmware-tanzu/community-edition/addons/packages/cert-manager/1.8.0/test/e2e/cert_manager_test.go:31
+ kubectl apply -f fixtures/cert-manager-resources.yaml
namespace/cert-manager-test created
issuer.cert-manager.io/test-selfsigned created
certificate.cert-manager.io/selfsigned-cert created

+ kubectl delete namespace cert-manager-test
namespace "cert-manager-test" deleted

• [SLOW TEST:6.804 seconds]
cert-manager Package E2E Test
/Users/amansha/go/src/github.com/vmware-tanzu/community-edition/addons/packages/cert-manager/1.8.0/test/e2e/cert_manager_test.go:16
  works
  /Users/amansha/go/src/github.com/vmware-tanzu/community-edition/addons/packages/cert-manager/1.8.0/test/e2e/cert_manager_test.go:31
------------------------------
STEP: cleaning up cert-manager package
+ tanzu package installed delete cert-manager --poll-interval 10s --poll-timeout 20m --namespace default --yes

STEP: validating the cert-manager package install no longer exists
+ kubectl -n default get packageinstall cert-manager
Error from server (NotFound): packageinstalls.packaging.carvel.dev "cert-manager" not found

STEP: cleaning up cert-manager namespace
+ kubectl delete namespace cert-manager
Error from server (NotFound): namespaces "cert-manager" not found

Ran 1 of 1 Specs in 64.841 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS

Ginkgo ran 1 suite in 1m6.879488638s
Test Suite Passed