Kubeflow installation failed in OpenShift cluster

[tppalani]

Validation Checklist

• [X] Is this a Kubeflow issue?
• [X] Are you posting in the right repository ?
• [X] Did you follow the installation guide https://github.com/kubeflow/manifests?tab=readme-ov-file ?
• [X] Is the issue report properly structured and detailed with version numbers?
• [X] Is this for Kubeflow development ?
• [X] Would you like to work on this issue?
• [X] Join our slack channel using wg-manifests.

Version

1.9

Describe your issue

I'm trying to install kubeflow manifest in mu open shift cluster after the installation we have identified we pods are not in running state, i will list of the pods with logs in below.

oc logs katib-db-manager-676ffb7c8c-4sj8m
 1 db.go:32] Using MySQL
1 connection.go:40] Ping to Katib db failed: dial tcp 1.30.9.34:3306: i/o timeout
1 connection.go:40] Ping to Katib db failed: dial tcp 1.30.9.34:3306: i/o timeout
1 connection.go:40] Ping to Katib db failed: dial tcp 1.30.9.34:3306: i/o timeout
1 connection.go:40] Ping to Katib db failed: dial tcp 1.30.9.34:3306: i/o timeout
1 connection.go:40] Ping to Katib db failed: dial tcp 1.30.9.34:3306: i/o timeout
1 connection.go:40] Ping to Katib db failed: dial tcp 1.30.9.34:3306: i/o timeout
1 connection.go:40] Ping to Katib db failed: dial tcp 1.30.9.34:3306: i/o timeout
1 connection.go:40] Ping to Katib db failed: dial tcp 1.30.9.34:3306: i/o timeout
1 connection.go:40] Ping to Katib db failed: dial tcp 1.30.9.34:3306: i/o timeout
1 connection.go:40] Ping to Katib db failed: dial tcp 1.30.9.34:3306: i/o timeout
1 connection.go:40] Ping to Katib db failed: dial tcp 1.30.9.34:3306: i/o timeout
1 connection.go:40] Ping to Katib db failed: dial tcp 1.30.9.34:3306: i/o timeout
1 connection.go:40] Ping to Katib db failed: dial tcp 1.30.9.34:3306: i/o timeout
1 main.go:104] Failed to open db connection: DB open failed: Timeout waiting for DB conn successfully opened.

oc logs profiles-deployment-6f85c8f4d4-cq98q
Error from server (BadRequest): container "kfam" in pod "profiles-deployment-6f85c8f4d4-cq98q" is waiting to start: PodInitializing

Steps to reproduce the issue

Just looked at the case on this page to see if any issues have been reported similar to mine.

Put here any screenshots or videos (optional)

No response