Try out GKE Autopilot - Githubissues

sroettger commented 3 years ago

https://cloud.google.com/blog/products/containers-kubernetes/introducing-gke-autopilot

sirdarckcat commented 3 years ago

https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview#host_options_restrictions

gcsfuse uses hostpath mapping in write mode.

sroettger commented 3 years ago

we had an alternative to hostpath but that was too expensive, right?

On Fri, Mar 19, 2021 at 7:55 AM sirdarckcat @.***> wrote:

https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview#host_options_restrictions

gcsfuse uses hostpath mapping in write mode.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/google/kctf/issues/259#issuecomment-802601572, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEM2JHSK7CBES6VGAUOQ4JTTELYPPANCNFSM4ZB3R2IA .

sirdarckcat commented 3 years ago

it also uses privileged https://github.com/google/kctf/blob/v1.0/kctf-operator/pkg/resources/daemon-gcsfuse.go - can we use fuse without priv?

sirdarckcat commented 3 years ago

https://lwn.net/Articles/742138/?

sirdarckcat commented 3 years ago

https://github.com/docker/for-linux/issues/321

sirdarckcat commented 3 years ago

to use fuse without priv we would need this: https://github.com/moby/moby/pull/41880

apiVersion: apps/v1
kind: Deployment
metadata:
  name: foo-deployment
  labels:
    app: foo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: foo
  template:
    metadata:
      labels:
        app: foo
      annotations:
        container.apparmor.security.beta.kubernetes.io/foo: unconfined
    spec:
      containers:
      - name: foo
        image: ubuntu:latest
        command: ['/bin/bash']
        args: ["-c", "sleep 1d"]
        securityContext:
          procMount: UnmaskedProcMount
          seccompProfile:
            type: Unconfined
        volumeMounts:
          - name: dev-dir
            mountPath: /dev
      volumes:
        - name: dev-dir
          hostPath:
            path: /dev

I guess we need to choose between autopilot and gcsfuse.

sroettger commented 3 years ago

In that case we should stick with gcsfuse unless we find some good alternative. Maybe revisit in 6-12 months?

On Fri, Mar 19, 2021 at 12:19 PM sirdarckcat @.***> wrote:

to use fuse without priv we would need this: moby/moby#41880 https://github.com/moby/moby/pull/41880

apiVersion: apps/v1kind: Deploymentmetadata: name: foo-deployment labels: app: foospec: replicas: 1 selector: matchLabels: app: foo template: metadata: labels: app: foo annotations: container.apparmor.security.beta.kubernetes.io/foo: unconfined spec: containers:

name: foo image: ubuntu:latest command: ['/bin/bash'] args: ["-c", "sleep 1d"] securityContext: procMount: UnmaskedProcMount seccompProfile: type: Unconfined volumeMounts:

name: dev-dir mountPath: /dev volumes:

name: dev-dir hostPath: path: /dev

I guess we need to choose between autopilot and gcsfuse.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/google/kctf/issues/259#issuecomment-802759766, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEM2JHUBN5QPNAVMBPFFMCTTEMXM3ANCNFSM4ZB3R2IA .

sirdarckcat commented 3 years ago

lets do that, yea. I'll leave this open.

eljeffeg commented 2 years ago

Curious if there were any updates or progress on this? Autopilot looks pretty cool for a Kubernetes newb.