ctron
commented
4 years ago Taking a closer look, this is some weird "fails to list" issue, rather than a permission issue.
Closed ctron closed 3 years ago
ctron
commented
4 years ago Taking a closer look, this is some weird "fails to list" issue, rather than a permission issue.
medyagh
commented
4 years ago @ctron I am curious, could you share the steps to create the PVC so I could replicate this issue myself?
if this is a permission issue, do you mind trying with
minikube start --alsologtostderr --wait=all
bonifaido
commented
4 years ago It looks like the storage-provisioner image (v1.8.1) that comes from the addon is quite old, and it doesn't work well with more recent versions of Kubernetes. The error message says the error is coming from the github.com/r2d4/external-storage package, which this repository doesn't contain anymore at v1.11.0.
If I change the image to gcr.io/k8s-minikube/storage-provisioner:latest (and apply some necessary RBAC rules, since the new image requires a broader scope of permissions) all PVCs are coming up correctly.
bonifaido
commented
4 years ago 
mostafa-eltaher
commented
4 years ago It looks like the storage-provisioner image (
v1.8.1) that comes from the addon is quite old, and it doesn't work well with more recent versions of Kubernetes. The error message says the error is coming from thegithub.com/r2d4/external-storagepackage, which this repository doesn't contain anymore atv1.11.0.If I change the image to
gcr.io/k8s-minikube/storage-provisioner:latest(and apply some necessary RBAC rules, since the new image requires a broader scope of permissions) all PVCs are coming up correctly.
That worked for me, in particular, the following commands fixed the issue on minikube 1.12.1
#minikube delete
docker pull gcr.io/k8s-minikube/storage-provisioner:latest
docker tag gcr.io/k8s-minikube/storage-provisioner:latest gcr.io/k8s-minikube/storage-provisioner:v1.8.1
minikube start
# Taken from: https://github.com/kubernetes/minikube/blob/e5cfa386b1d978096d3e8d917f5ed07b960a457c/deploy/addons/storage-provisioner/storage-provisioner.yaml.tmpl
cat <<EOF | kubectl apply -f -
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: system::leader-locking-storage-provisioner
namespace: kube-system
labels:
addonmanager.kubernetes.io/mode: EnsureExists
rules:
- apiGroups:
- ""
resources:
- endpoints
verbs:
- watch
- apiGroups:
- ""
resourceNames:
- k8s.io-minikube-hostpath
resources:
- endpoints
verbs:
- get
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: leader-locking-storage-provisioner
namespace: kube-system
labels:
addonmanager.kubernetes.io/mode: EnsureExists
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: system::leader-locking-storage-provisioner
subjects:
- kind: ServiceAccount
name: storage-provisioner
namespace: kube-system
---
apiVersion: v1
kind: Endpoints
metadata:
name: k8s.io-minikube-hostpath
namespace: kube-system
labels:
addonmanager.kubernetes.io/mode: Reconcile
EOF
priyawadhwa
commented
3 years ago Hey @ctron in the past few months we've upgraded the storage provisioner and fixed some bugs. Could you try:
Thanks.
ctron
commented
3 years ago Hey @ctron in the past few months we've upgraded the storage provisioner and fixed some bugs. Could you try:
1. Upgrading to the latest version of minikube, 1.13.1 2. Let us know if you still see this issue with the latest version?Thanks.
Will do.
tstromberg
commented
3 years ago @ctron - any update? We're now on minikube v1.4.0 -- though I don't think the storage provisioner has changed since v1.13.1.
medyagh
commented
3 years ago @ctron do you still have this issue with latest version of minikube?
ctron
commented
3 years ago I am now on minikube version: v1.14.0 … and it seems to work.
ctron
commented
3 years ago I just checked, looks like 0.15.1 is the most recent. I will upgrade and test in the next few days. But I think it is ok to close this issue.
tstromberg
commented
3 years ago Thanks for the update!
Steps to reproduce the issue:
minikube start --cpus 4 --memory 8192Full output of failed command:
Full output of
minikube startcommand used, if not already included:Logs of the storage provisioner:
Optional: Full output of
minikube logscommand: