GoogleCloudPlatform / gcsfuse

A user-space file system for interacting with Google Cloud Storage
https://cloud.google.com/storage/docs/gcs-fuse
Apache License 2.0
2.05k stars 426 forks source link

docs: add installation instructions for Dockerfile #400

Open grayside opened 4 years ago

grayside commented 4 years ago

Thank you for the installation instructions which cover so many operating systems. Container environments are sufficiently non-standard that it can be tricky to work out the best way to install and start gcsfuse.

Some of the elements of the current documentation that gave me trouble in a buster-slim container image include:

Proposed Addition

Documentation on working with containers covering:

I ended up adding curl and fuse to an apt-get install block, then inserting the following:

ENV GCSFUSE_VERSION 0.27.0
RUN curl -L -O https://github.com/GoogleCloudPlatform/gcsfuse/releases/download/v${GCSFUSE_VERSION}/gcsfuse_${GCSFUSE_VERSION}_amd64.deb \
 && dpkg --install gcsfuse_${GCSFUSE_VERSION}_amd64.deb \
 && rm -Rf gcsfuse_${GCSFUSE_VERSION}_amd64.deb
bjornleffler commented 4 years ago

This is a great question. Very relevant since Docker / k8s usage and interest seems to be increasing.

One tricky aspect of running gcsfuse is probably to get all the permissions right. I'm guessing a docker container would need some non-default privileges to mount a fuse file system.

I'll dig into this.

grayside commented 4 years ago

I'm told --privileged works. I was able to run it with --cap-add SYS_ADMIN --device /dev/fuse

pgillet commented 3 years ago

Hi, I was not able to run my Ubuntu-based image with --cap-add SYS_ADMIN --device /dev/fuse. It works like a charm with --privileged though, and with root or non-root user. But I would like to avoid this option.

My Dockerfile:

FROM ubuntu:18.04

RUN apt-get update
RUN apt-get install -y gnupg lsb-release wget

RUN lsb_release -c -s > /tmp/lsb_release
RUN GCSFUSE_REPO=$(cat /tmp/lsb_release); echo "deb http://packages.cloud.google.com/apt gcsfuse-$GCSFUSE_REPO main" | tee /etc/apt/sources.list.d/gcsfuse.list
RUN wget -O - https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -

RUN apt-get update
RUN apt-get install -y gcsfuse

My test:

docker run -it --rm --device /dev/fuse --cap-add SYS_ADMIN -v /path/in/host/to/key.json:/path/to/key.json -e GOOGLE_APPLICATION_CREDENTIALS=/path/to/key.json my_image:0.1 /bin/bash

In the running container:

mkdir /root/gruik
gcsfuse bucket_name /root/gruik/

The result:

Using mount point: /root/gruik
Opening GCS connection...
Mounting file system...
daemonize.Run: readFromProcess: sub-process: mountWithArgs: mountWithConn: Mount: mount: permission denied

Am I missing something? Thanks

jonathantito commented 3 years ago

You need a privileged container.

If you are using docker-compose: privileged: true

If you are using docker run: --privileged

And this is what I have in the Dockerfile:

RUN apt-get update RUN apt-get install -y gnupg lsb-release wget

RUN lsb_release -c -s > /tmp/lsb_release RUN GCSFUSE_REPO=$(cat /tmp/lsb_release); echo "deb http://packages.cloud.google.com/apt gcsfuse-$GCSFUSE_REPO main" | tee /etc/apt/sources.list.d/gcsfuse.list RUN wget -O - https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -

RUN apt-get update RUN apt-get install -y gcsfuse

omiguelpinheiro commented 1 year ago

Have you found a way to install gcsfuse on alpine?

sethiay commented 1 year ago

@omiguelpinheiro , We recommend using Ubuntu and Debian OS for GCSFuse. Couple of things to keep in mind while using GCSFuse in docker container:

  1. fuse is installed.
  2. container is run in --privileged mode (https://github.com/GoogleCloudPlatform/gcsfuse/blob/master/docs/playbook.md#gcsfuse-fails-with-docker-container).

Also, could you please share the issue that you are facing ?

sethiay commented 1 year ago

For now, we recommend using this Dockerfile to create Ubuntu/Debain/Distroless images with gcsfuse installed. Please let us know here if you face any issues with the images/Dockerfile.