search

DataBiosphere / gitlab-in-google

Google Cloud Based Gitlab CI/CD Platform with Kubernetes in Terraform.
0 stars 0 forks source link

Jobs are failing waiting for Kubernetes pods #2

Open xbrianh opened 4 years ago

xbrianh commented 4 years ago

This is occurring frequently for bdcat_notebooks, which defines several parallel jobs. see: https://biodata-integration-tests.net/databiosphere/bdcat_notebooks/-/pipelines/1429

Waiting for pod kube-system/runner-5vnamnhm-project-7-concurrent-02f2ww to be running, status is Pending
Waiting for pod kube-system/runner-5vnamnhm-project-7-concurrent-02f2ww to be running, status is Pending
Waiting for pod kube-system/runner-5vnamnhm-project-7-concurrent-02f2ww to be running, status is Pending
Waiting for pod kube-system/runner-5vnamnhm-project-7-concurrent-02f2ww to be running, status is Pending
Waiting for pod kube-system/runner-5vnamnhm-project-7-concurrent-02f2ww to be running, status is Pending
Waiting for pod kube-system/runner-5vnamnhm-project-7-concurrent-02f2ww to be running, status is Pending
Waiting for pod kube-system/runner-5vnamnhm-project-7-concurrent-02f2ww to be running, status is Pending
Waiting for pod kube-system/runner-5vnamnhm-project-7-concurrent-02f2ww to be running, status is Pending
Waiting for pod kube-system/runner-5vnamnhm-project-7-concurrent-02f2ww to be running, status is Pending
Waiting for pod kube-system/runner-5vnamnhm-project-7-concurrent-02f2ww to be running, status is Pending
Waiting for pod kube-system/runner-5vnamnhm-project-7-concurrent-02f2ww to be running, status is Pending
Waiting for pod kube-system/runner-5vnamnhm-project-7-concurrent-02f2ww to be running, status is Pending
ERROR: Job failed (system failure): timed out waiting for pod to start
xbrianh commented 3 years ago

observed again: https://biodata-integration-tests.net/xbrianh/terra-notebook-utils/-/jobs/79257

xbrianh commented 3 years ago

again: https://biodata-integration-tests.net/xbrianh/terra-notebook-utils/-/jobs/80142

xbrianh commented 3 years ago

Some correlation with other jobs? not sure: https://biodata-integration-tests.net/xbrianh/terra-notebook-utils/-/jobs/80148