Local minikube tutorial failure

[bgalvao]

I am trying to follow the tutorial Set up your local minikube machine learning architecture for experimenting hands-on with the components of OpenMLOps. Unfortunately, I am not able to follow all the instructions to the end.

Disclaimer: I am new to Terraform and Kubernetes.

To begin with, I tried to start minikube in two ways,

1. with minikube start --kubernetes-version=v1.17.17
2. and without a specified kubernetes version.

In case 1., I could not get minikube to start running. Here is the entire output of the command.

In the latter (without specifying the version), I believe minikube defaults to v1.22.2 as per the output I get from kubectl version. In this case, I am able to get minikube running and start a tunnel as well:

Status:
        machine: minikube
        pid: 10105
        route: 10.96.0.0/12 -> 192.168.49.2
        minikube: Running
        services: [mlflow]
    errors: 
                minikube: no errors
                router: no errors
                loadbalancer emulator: no errors

All ok until I get to the step terraform apply -var-file=my_vars.tfvars, which outputs the following at the end (the only 3 changes not able to complete):

module.ambassador[0].helm_release.ambassador[0]: Creating...
module.dask-jupyterhub.helm_release.dask-jupyterhub: Creating...
module.prefect-server.helm_release.prefect-server: Creating...
╷
│ Error: failed to install CRD crds/filter.yaml: unable to recognize "": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
│ 
│   with module.ambassador[0].helm_release.ambassador[0],
│   on modules/ambassador/main.tf line 1, in resource "helm_release" "ambassador":
│    1: resource "helm_release" "ambassador" {
│ 
╵
╷
│ Error: failed to install CRD crds/daskclusters.yaml: unable to recognize "": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
│ 
│   with module.dask-jupyterhub.helm_release.dask-jupyterhub,
│   on modules/dask-jupyterhub/main.tf line 1, in resource "helm_release" "dask-jupyterhub":
│    1: resource "helm_release" "dask-jupyterhub" {
│ 
╵
╷
│ Error: unable to build kubernetes objects from release manifest: unable to recognize "": no matches for kind "RoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
│ 
│   with module.prefect-server.helm_release.prefect-server,
│   on modules/prefect-server/main.tf line 1, in resource "helm_release" "prefect-server":
│    1: resource "helm_release" "prefect-server" {
│ 
╵

Why is this happening?

[pipatth]

Hi @bgalvao,

I think the Ambassador version that we are using depends on Kubernetes 1.17. Let me try to replicate spinning up minikube with a specific version. I'll get back to you.

[pipatth]

I can replicate the issue using minikube v1.23.0. The tutorial doesn't work on kubernetes 1.22.1. It works with 1.17.17.

I am not sure why you can't install v1.17.17. The best course of action is to try other kubernetes versions (e.g. v1.18) and see if you can install it.

[bgalvao]

Ok, I went backwards from 1.22 and I got to make it work with minikube start --kubernetes-version=v1.21.5 ✅🥳️

---

The following are comments for anyone that may struggle

• I also ran terraform init -upgrade in the project root.
• I should mention that I ran terraform apply -var-file=my_vars.tfvars twice, since the first time timed out building prefect-server.

---

For future reference and summarizing a working combination of versions:

• Docker version 20.10.9, build c2ea9bc90b
• Terraform v1.0.8
• kubectl server version v1.21.5 and client version v1.22.2, which includes:
  • minikube version: v1.23.2
• minikube start --kubernetes-version=v1.21.5

[bgalvao]

closing because everything is working now.