SFTP on GCSFuse mount with Chroot

[xyoun]

I'm trying to mount a bucket on my sftp server and only allow specific sftp users access(read&write) only the data in this bucket.

according to sftp setup guide and document, https://wiki.archlinux.org/index.php/SFTP_chroot we need:

1. make chroot dir's owner to root
2. make chroot dir's mode to 755 (only u has write permission)

I tried to have these parameters to mount: rw,allow_other,dir_mode=0755,file_mode=0775,uid=root

but the problems is, the sftp user can only read the data, they're not allow to write. it seems the dir_mode works for both bucket dir and all subdirs.

I also tried to use mount --bind to mount it to a 775 directory, but the gcsfuse will overwrite dir's mode and make it back to 755.

so is there anyway to make the the sftp user chroot into the bucket and have read & write access?

[bjornleffler]

To create a file on a unix file system, you also need write permission on the directory. What happens if you change the dir_mode to 775?

Make sure the group ids of the [directory] and the [sftp user] match.

[xyoun]

@bjornleffler if I change the dir_mode to 775, the sftp user will not able to login with log error in this way:

Aug 19 03:46:17 sftp-prod01 sshd[19363]: fatal: bad ownership or modes for chroot directory "/mnt/buckets/xxx" [postauth]
Aug 19 03:46:17 sftp-prod01 sshd[19363]: pam_unix(sshd:session): session closed for user xxx```

[bjornleffler]

That looks like a problem with the sftp server, or its configuration.

I'd try writing a file without the sftp server. Change to the sftp user and try to write some files in that directory.

[xyoun]

it's kind like the restriction by ssh for security reason: https://serverfault.com/questions/418931/sftp-fatal-bad-ownership-or-modes-for-chroot-directory-ubuntu-12-04/418937

I'm currently try to setup a service like https://aws.amazon.com/sftp/, since GCP does not has this service so I need to build by myself.

[bjornleffler]

Thanks for those references. I'll try this tomorrow.

I'm wondering if one directory per authorised user could be a workaround.

[bjornleffler]

I tried this today and got a similar security error message in /var/log/auth.log

sshd[3077]: Authentication refused: bad ownership or modes for directory Sftp doesn't allow the chroot directory to be group or world writable (775/777), bus sub directories can be. I would try mounting the GCS bucket one level down from the chroot directory. For example /sftp as the chroot directory. Then mount gs://bucket123 with GCS Fuse at /sftp/bucket123.

[xyoun]

@bjornleffler yeah! it works in this way. I'm wondering if gcsfuse can have a parameter like "subdir_mode" to allow setting the bucket permission to 755 and other directories to 775?

here is a scenario in our company: we provide our clients a sftp address so they can upload data to our buckets. right now we have AWS SFTP(https://aws.amazon.com/sftp/) which works perfectly. they run sftp client@sftp_address and put file. if we can only mount GCS's bucket in a sub directory, then every client will see all clients bucket in this sftp. this is not a good idea.

now I have a workaround:

bucket_name /mnt/bucket gcsfuse rw,allow_other,dir_mode=0750,file_mode=0750,uid=root,gid=sftpgroup,key_file=/mnt/key.json
bucket_name /mnt/bucket/data gcsfuse rw,allow_other,dir_mode=0770,file_mode=0770,uid=root,gid=sftpgroup,nonempty,only_dir=data,key_file=/mnt/key.json

but in this way we can only write in one directory.

[bjornleffler]

Got it. I see what you're trying to do. Sftp normally starts a user in their home directory. This is a natural redirect on sftp login.

One UNIX trick to have private subfolder is to have the parent folder 711, owned by some super user, and then subfolders with permissions 700, owned by the individual users. Any user can cd to the root folder, but cannot list it, as they don't have read permissions. Any user can read and write to their directory, but not read/write anywhere else.

I might play a bit more with this tomorrow, if I have time.

[richardmilnerwatts]

A bit late, but I did a load of work on using gcsfuse to back up a SFTP container running in K8s, we ran an ansible role on container start to define users and their mounts to a GCS bucket via fuse.

Snippet below if anyone needs:

- name: "Create sftp user: {{ sftp_instance.username }}"
  user:
    name: "{{ sftp_instance.username }}"
    createhome: no
    home: "{{ sftp_init_k8s__home_folder }}/{{ sftp_instance.username }}"
    uid: "{{ sftp_instance.user_id }}"
    group: "{{ sftp_init_k8s__group_name }}"
    shell: /sbin/nologin
    state: present
    password: "{{ sftp_instance.password }}"

- name: "Create home directory for user {{ sftp_instance.username }}"
  file: 
    path: "{{ sftp_init_k8s__home_folder }}/{{ sftp_instance.username }}"
    owner: "root"
    group: "{{ sftp_init_k8s__group_name }}"
    mode: 0755
    state: directory

- name: "Create bucket mount target folder: {{ item.target_folder }}"
  file: 
    path: "{{ sftp_init_k8s__home_folder }}/{{ sftp_instance.username }}/{{ item.target_folder }}"
    owner: "{{ sftp_instance.username }}"
    group: "{{ sftp_init_k8s__group_name }}"
    mode: 0755
    state: directory
  with_items: "{{ sftp_instance.mount_details }}"

## Should the user have a public key it is to be added to the authorized keys file for that user
- name: "Add authorized keys for user"
  authorized_key:
    user: "{{ sftp_instance.username }}"
    key: "{{ item }}"
    path: "{{ sftp_init_k8s__home_folder }}/{{ sftp_instance.username }}/.ssh/authorized_keys"
  with_items: "{{ sftp_instance.authorized_keys }}"
  when:
    - sftp_instance.authorized_keys|length > 0

# Mount the bucket using GCSfuse
#  https://github.com/GoogleCloudPlatform/gcsfuse
- name: "Mount the GCP bucket using GCSFuse as folder: {{ item.target_folder }}"
  shell: "/usr/bin/gcsfuse --dir-mode 755 --file-mode 644 --uid {{ sftp_instance.user_id }} --gid {{ sftp_init_k8s__group_id }} --only-dir {{ item.bucket_folder }} -o allow_other --implicit-dirs {{ sftp_init_k8s__gcp_bucket_name }} {{ sftp_init_k8s__home_folder }}/{{ sftp_instance.username }}/{{ item.target_folder }}"
  with_items: "{{ sftp_instance.mount_details }}"

Of particular interest were the flags "allow_other" and "implicit-dirs".

[bjornleffler]

Wow, that's very impressive. I thought you would have had to change the source code of sftp to make this work.

[american-psycho]

here's my ansible structure:

gcs_fuse: 
- username: "jimbo"
  group: sftp
  mounts:
  - bucket: "gcloud-bucketname.appspot.com"
    mount_point: "/home/jimbo/users"
    bucket_dir: "users"
  - bucket: "gcloud-bucketname.appspot.com"
    mount_point: "/home/jimbo/dealer"
    bucket_dir: "dealer"
  - bucket: "gcloud-bucketname.appspot.com"
    mount_point: "/home/jimbo/registrations"
    bucket_dir: "registrations"

and my ansible command for the fstab persistent mount

- name: adding mount point folders to fstab
  mount: >
    path="{{ item.1.mount_point }}"
    src="{{ item.1.bucket }}"
    opts="rw,allow_other,uid={{item.0.username}},gid={{item.0.group}},file_mode=0644,dir_mode=0755,only_dir={{ item.1.bucket_dir }},implicit_dirs"
    fstype="gcsfuse"
    state=mounted
  with_subelements:
  - "{{gcs_fuse}}"
  - mounts

[jbielick]

This is an older thread, but I thought it was worth pointing out the current docs recommend against mounting with allow_other if possible.

As a security measure, fuse itself restricts file system access to the user who mounted the file system (cf. fuse.txt). For this reason, gcsfuse by default shows all files as owned by the invoking user. Therefore you should invoke gcsfuse as the user that will be using the file system, not as root.

If you know what you are doing, you can override these behaviors with the allow_other mount option supported by fuse and with the --uid and --gid flags supported by gcsfuse. Be careful, this may have security implications!

https://github.com/GoogleCloudPlatform/gcsfuse/blob/master/docs/mounting.md#access-permissions

I'm curious if (based on the ansible example) mounting as the sftp user themself with become: {{ sftp_instance.username }} or something would work and then allow_other,gid,uid would not be necessary? Haven't tried it yet. Perhaps using ansible mount and writing to fstab directly accomplishes the same thing?

[DesmondH0]

I am late to this party. What I was trying to do is mount the bucket with gcsfuse and the sftp user would login to the root of that bucket. So that I can keep all the sftp having their separated bucket.

1. I mount the bucket to /home/$user/files:

   gcsfuse --key-file cred.json $GCS_BUCKET /home/$user/files

2. Then the hack would be on SSHD level:

   Subsystem sftp internal-sftp
   ForceCommand internal-sftp -l INFO -d /files
   ChrootDirectory %h

So the user would be automatically switch into /files after login. Although they could cd to upper level, but cannot write anything since the /home/$user level is owned by root

[avidullu]

Seems there are enough examples in the discussion which recommend solutions and workarounds for the original issue and @xyoun also was able to resolve their issue. Closing this out but feel free to reopen in case more investigation needs to be done.