Closed Chancebair closed 5 years ago
Apparently this is us hitting request limits for unauthenticated users to the kubeflow github
Something to try is authenticating via a token first: https://help.github.com/en/articles/creating-a-personal-access-token-for-the-command-line
Looks like a missed failure in _install_kubeflow_operators from baidriver
==> Installing kubeflow operators
12:52:11
level=info msg="Using context \"eks_benchmark-cluster\" from kubeconfig file \"/root/.bai/kubeconfig\""
12:52:11
level=info msg="Creating environment \"default\" with namespace \"default\", pointing to \"version:v1.12.6\" cluster at address \"https://A66C4AD834C534191070EDAF6D16FAF7.yl4.us-west-2.eks.amazonaws.com\""
12:52:19
level=info msg="Generating ksonnet-lib data at path '/root/.bai/kubeflow-ks-app/ks_app/lib/ksonnet-lib/v1.12.6'"
12:52:20
-> Creating kubeflow namespace
12:52:20
namespace/kubeflow created
12:52:20
-> Applying kubeflow stuff
12:52:22
level=info msg="Using context \"eks_benchmark-cluster\" from kubeconfig file \"/root/.bai/kubeconfig\""
12:52:22
level=info msg="Creating environment \"kubeflow\" with namespace \"kubeflow\", pointing to \"version:v1.12.6\" cluster at address \"https://A66C4AD834C534191070EDAF6D16FAF7.yl4.us-west-2.eks.amazonaws.com\""
12:52:32
level=info msg="Retrieved 6 files"
12:52:34
level=info msg="Retrieved 5 files"
12:52:34
level=info msg="Writing component at '/root/.bai/kubeflow-ks-app/ks_app/components/mpi-operator.jsonnet'"
12:52:35
level=info msg="Writing component at '/root/.bai/kubeflow-ks-app/ks_app/components/mxnet-operator.jsonnet'"
12:52:35
-> Kubeflow should be already installed, re-applying configuration
12:52:45
[ERROR] Failed with exit code: 137
12:52:45
/work/baictl/drivers/aws/baidriver: line 222: 770 Killed ks apply kubeflow --component mpi-operator
Closing in favor of dupe https://github.com/MXNetEdge/benchmark-ai/issues/424