squat / generic-device-plugin

A Kubernetes device plugin to schedule generic Linux devices
Apache License 2.0
210 stars 23 forks source link

Cannot mount fuse on in minikube: fusermount: fuse device not found #56

Open skoulouzis opened 1 year ago

skoulouzis commented 1 year ago

Hi, Thanks for making this plug-in. I want to use rclone to mount several cloud storage (s3, Google Drive, etc.) in a Pod in k8s. To test it I'm using Minikube where I installed the plugin using:

kubectl apply -f https://raw.githubusercontent.com/squat/generic-device-plugin/main/manifests/generic-device-plugin.yaml

Then I created a pod to test the mount:

apiVersion: v1
kind: Pod
metadata:
  name: ubuntu
spec:
  containers:
  - name: ubuntu
    image: ubuntu:latest
    command: ["/bin/sh", "-c", "apt-get update && apt-get install -y rclone fuse && mkdir /mnt/my_remote_bucket && tail -f /dev/null"]
    resources:
    limits:
      squat.ai/fuse: 1
      securityContext:
        capabilities:
          add: ["SYS_ADMIN"]
    # mount the rclone config file
    volumeMounts:
    - name: rclone-config
      mountPath: /root/.config/rclone/rclone.conf
      subPath: rclone.conf
  volumes:
  - name: rclone-config
    configMap:
      name: rclone-config
      items:
      - key: rclone.conf
        path: rclone.conf

As soon as the pod starts I run in the pod:

rclone mount s3:bucket /mnt/my_remote_bucket --no-check-certificate --allow-other --allow-non-empty --vfs-cache-mode writes

The command returns:

2023/11/15 19:43:08 mount helper error: fusermount: fuse device not found, try 'modprobe fuse' first
2023/11/15 19:43:08 Fatal error: failed to mount FUSE fs: fusermount: exit status 1

Any idea why I may get this error?

squat commented 10 months ago

Hi @skoulouzis have you had any luck? This seems like a problem with the pod's configuration. If the pod starts, then it means that the Kubernetes scheduler was able to find a node with FUSE devices and the Kubelet was able to have one allocated.

Questions: do you exec the rclone mount command manually? Why is that, if I may ask? Why not put it after the alt-get install instead of the tail? Are you exec-ing in the same container? Or a different container in the pod?