I did installation of vanilla v1.8.0 on private AKS 1.29.1 and found that following secrets are not created
MountVolume.SetUp failed for volume "cert" : secret "webhook-server-cert" not found
MountVolume.SetUp failed for volume "cert" : secret "kserve-webhook-server-cert" not found
MountVolume.SetUp failed for volume "cert" : secret "katib-webhook-cert" not found
MountVolume.SetUp failed for volume "webhook-cert" : secret "webhook-certs" not found
On verifying the logs of cert-manager-cainjector i found following errors
I0912 06:13:06.545486 1 controller.go:219] "cert-manager: Starting workers" controller="validatingwebhookconfiguration" controllerGroup="admissionregistration.k8s.io" controllerKind="ValidatingWebhookConfiguration" worker count=1
E0912 06:13:06.546236 1 sources.go:106] "cert-manager: unable to fetch associated certificate" err="Certificate.cert-manager.io \"admission-webhook-cert\" not found" kind="mutatingwebhookconfiguration" kind="mutatingwebhookconfiguration" name="admission-webhook-mutating-webhook-configuration" certificate="kubeflow/admission-webhook-cert"
I0912 06:13:06.546268 1 reconciler.go:118] "cert-manager: could not find any ca data in data source for target" kind="mutatingwebhookconfiguration" kind="mutatingwebhookconfiguration" name="admission-webhook-mutating-webhook-configuration"
E0912 06:13:06.546326 1 sources.go:106] "cert-manager: unable to fetch associated certificate" err="Certificate.cert-manager.io \"kfp-cache-cert\" not found" kind="mutatingwebhookconfiguration" kind="mutatingwebhookconfiguration" name="cache-webhook-kubeflow" certificate="kubeflow/kfp-cache-cert"
I0912 06:13:06.546337 1 reconciler.go:118] "cert-manager: could not find any ca data in data source for target" kind="mutatingwebhookconfiguration" kind="mutatingwebhookconfiguration" name="cache-webhook-kubeflow"
E0912 06:13:06.546378 1 sources.go:106] "cert-manager: unable to fetch associated certificate" err="Certificate.cert-manager.io \"katib-webhook-cert\" not found" kind="mutatingwebhookconfiguration" kind="mutatingwebhookconfiguration" name="katib.kubeflow.org" certificate="kubeflow/katib-webhook-cert"
I0912 06:13:06.546391 1 reconciler.go:118] "cert-manager: could not find any ca data in data source for target" kind="mutatingwebhookconfiguration" kind="mutatingwebhookconfiguration" name="katib.kubeflow.org"
E0912 06:13:06.546424 1 sources.go:106] "cert-manager: unable to fetch associated certificate" err="Certificate.cert-manager.io \"serving-cert\" not found" kind="mutatingwebhookconfiguration" kind="mutatingwebhookconfiguration" name="inferenceservice.serving.kserve.io" certificate="kubeflow/serving-cert"
I0912 06:13:06.546445 1 reconciler.go:118] "cert-manager: could not find any ca data in data source for target" kind="mutatingwebhookconfiguration" kind="mutatingwebhookconfiguration" name="inferenceservice.serving.kserve.io"
E0912 06:13:06.546478 1 sources.go:106] "cert-manager: unable to fetch associated certificate" err="Certificate.cert-manager.io \"pvcviewer-serving-cert\" not found" kind="mutatingwebhookconfiguration" kind="mutatingwebhookconfiguration" name="pvcviewer-mutating-webhook-configuration" certificate="kubeflow/pvcviewer-serving-cert"
I0912 06:13:06.546490 1 reconciler.go:118] "cert-manager: could not find any ca data in data source for target" kind="mutatingwebhookconfiguration" kind="mutatingwebhookconfiguration" name="pvcviewer-mutating-webhook-configuration"
E0912 06:13:06.547142 1 sources.go:106] "cert-manager: unable to fetch associated certificate" err="Certificate.cert-manager.io \"pvcviewer-serving-cert\" not found" kind="customresourcedefinition" kind="customresourcedefinition" name="pvcviewers.kubeflow.org" certificate="kubeflow/pvcviewer-serving-cert"
I0912 06:13:06.547172 1 reconciler.go:118] "cert-manager: could not find any ca data in data source for target" kind="customresourcedefinition" kind="customresourcedefinition" name="pvcviewers.kubeflow.org"
I0912 06:13:06.551303 1 reconciler.go:142] "cert-manager: Updated object" kind="validatingwebhookconfiguration" kind="validatingwebhookconfiguration" name="cert-manager-webhook"
E0912 06:13:06.551407 1 sources.go:106] "cert-manager: unable to fetch associated certificate" err="Certificate.cert-manager.io \"serving-cert\" not found" kind="validatingwebhookconfiguration" kind="validatingwebhookconfiguration" name="inferencegraph.serving.kserve.io" certificate="kubeflow/serving-cert"
I0912 06:13:06.551430 1 reconciler.go:118] "cert-manager: could not find any ca data in data source for target" kind="validatingwebhookconfiguration" kind="validatingwebhookconfiguration" name="inferencegraph.serving.kserve.io"
E0912 06:13:06.551486 1 sources.go:106] "cert-manager: unable to fetch associated certificate" err="Certificate.cert-manager.io \"serving-cert\" not found" kind="validatingwebhookconfiguration" kind="validatingwebhookconfiguration" name="inferenceservice.serving.kserve.io" certificate="kubeflow/serving-cert"
I0912 06:13:06.551505 1 reconciler.go:118] "cert-manager: could not find any ca data in data source for target" kind="validatingwebhookconfiguration" kind="validatingwebhookconfiguration" name="inferenceservice.serving.kserve.io"
E0912 06:13:06.551542 1 sources.go:106] "cert-manager: unable to fetch associated certificate" err="Certificate.cert-manager.io \"katib-webhook-cert\" not found" kind="validatingwebhookconfiguration" kind="validatingwebhookconfiguration" name="katib.kubeflow.org" certificate="kubeflow/katib-webhook-cert"
I0912 06:13:06.551558 1 reconciler.go:118] "cert-manager: could not find any ca data in data source for target" kind="validatingwebhookconfiguration" kind="validatingwebhookconfiguration" name="katib.kubeflow.org"
E0912 06:13:06.551598 1 sources.go:106] "cert-manager: unable to fetch associated certificate" err="Certificate.cert-manager.io \"serving-cert\" not found" kind="validatingwebhookconfiguration" kind="validatingwebhookconfiguration" name="servingruntime.serving.kserve.io" certificate="kubeflow/serving-cert"
I0912 06:13:06.551631 1 reconciler.go:118] "cert-manager: could not find any ca data in data source for target" kind="validatingwebhookconfiguration" kind="validatingwebhookconfiguration" name="servingruntime.serving.kserve.io"
E0912 06:13:06.551673 1 sources.go:106] "cert-manager: unable to fetch associated certificate" err="Certificate.cert-manager.io \"serving-cert\" not found" kind="validatingwebhookconfiguration" kind="validatingwebhookconfiguration" name="trainedmodel.serving.kserve.io" certificate="kubeflow/serving-cert"
I0912 06:13:06.551692 1 reconciler.go:118] "cert-manager: could not find any ca data in data source for target" kind="validatingwebhookconfiguration" kind="validatingwebhookconfiguration" name="trainedmodel.serving.kserve.io"
E0912 06:13:06.551737 1 sources.go:106] "cert-manager: unable to fetch associated certificate" err="Certificate.cert-manager.io \"serving-cert\" not found" kind="validatingwebhookconfiguration" kind="validatingwebhookconfiguration" name="clusterservingruntime.serving.kserve.io" certificate="kubeflow/serving-cert"
I0912 06:13:06.551755 1 reconciler.go:118] "cert-manager: could not find any ca data in data source for target" kind="validatingwebhookconfiguration" kind="validatingwebhookconfiguration" name="clusterservingruntime.serving.kserve.io"
E0912 06:13:06.551810 1 sources.go:106] "cert-manager: unable to fetch associated certificate" err="Certificate.cert-manager.io \"pvcviewer-serving-cert\" not found" kind="validatingwebhookconfiguration" kind="validatingwebhookconfiguration" name="pvcviewer-validating-webhook-configuration" certificate="kubeflow/pvcviewer-serving-cert"
I0912 06:13:06.551828 1 reconciler.go:118] "cert-manager: could not find any ca data in data source for target" kind="validatingwebhookconfiguration" kind="validatingwebhookconfiguration" name="pvcviewer-validating-webhook-configuration"
I0912 06:13:06.552764 1 reconciler.go:142] "cert-manager: Updated object" kind="mutatingwebhookconfiguration" kind="mutatingwebhookconfiguration" name="cert-manager-webhook"
E0912 06:13:06.555537 1 sources.go:106] "cert-manager: unable to fetch associated certificate" err="Certificate.cert-manager.io \"serving-cert\" not found" kind="customresourcedefinition" kind="customresourcedefinition" name="inferenceservices.serving.kserve.io" certificate="kubeflow/serving-cert"
I0912 06:13:06.555564 1 reconciler.go:118] "cert-manager: could not find any ca data in data source for target" kind="customresourcedefinition" kind="customresourcedefinition" name="inferenceservices.serving.kserve.io"
I did installation of vanilla v1.8.0 on private AKS 1.29.1 and found that following secrets are not created
On verifying the logs of cert-manager-cainjector i found following errors