Shared storage across containers

[craig-willis]

I am trying to created a "shared" directory for my workshop and am stuck. Here is what I'd like to do "in theory": mount a shared space (e.g. folder, drive, etc.) where all the workshop participants can see (read/write) the data in it. The goal here is to actually have a single place for data that all participants will use and to also have a place for shared notebooks that are not pushed back up to Github. What's the fastest+best path to that solution?

[craig-willis]

The rook approach we're using for individual user data (ReadWriteOnce persistent volume claims) does not support shared volumes. We've been exploring using rook's shared volume support, but have come to the conclusion that simple NFS is even better. @bodom0015 has written something up on both NFS and Rook approaches for a different project.

Using the in-cluster NFS approach outlined in the Wiki, we can update the jupyterhub config to mount the extra volume:

singleuser:
  memory:
    guarantee: 1G
    limit: 8G
  storage:
    type: dynamic
    capacity: 10Gi
    dynamic:
      storageClass: rook-block
    extraVolumeMounts:
      - name: nfs-share
        mountPath: /shared
    extraVolumes:
      - name: nfs-share
        nfs:
          server: "10.105.209.51"
          path: "/"
          readOnly: false
  image:
    name: ndslabs/esiphub-notebook
    tag: latest

We're using the service IP here because Kubernetes internal DNS isn't accessible from the host. Eventually we might resolve this by modifying the host resolve.conf.

This will now show up as a `/shared' directory in the user's Jupyter instance.

[jreadey]

How will JupyterLab users have permissions to write to the volume? From the container the user is "jovyan" but this won't mean anything to the NFS share.

[craig-willis]

In the container the jovyan user has uid 1000, which currently maps to the ubuntu user on the host OS. All files will appear to be owned by jovyan across containers and by ubuntu on the host.

There are other ways to do this, but this should work for Keith's short-term need. I believe the NFS storage driver would support UID/GID, but that would need to map back to OS users. We've prototyped this via something like LDAP with PAM/sssd, but it's a bit more involved and wouldn't work well with Oauth.

[jreadey]

For many use-cases I suspect it would fine just having a read-only share for uid 1000.

[craig-willis]

Agreed. In this specific case, the idea is for workshop participants to be able to share things with each other outside of Github. For other workshops, we often have read-only mounts.

[kmaull-ucar]

Yes, we should be able to support both cases. Workshop hosts may want to load a bunch of local data for read-only scenarios, but we might also want to support the ability for shared spaces so workshop participants could drop files in a location (e.g. notebooks or output files) other participants could access.

Thanks again, for helping put this in place!

On Thu, Jun 14, 2018 at 1:10 PM, Craig Willis notifications@github.com wrote:

Agreed. In this specific case, the idea is for workshop participants to be able to share things with each other outside of Github. For other workshops, we often have read-only mounts.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/nds-org/esiphub/issues/8#issuecomment-397406486, or mute the thread https://github.com/notifications/unsubscribe-auth/AE_VQreE4AGPu4r1Ard8rowX9RoPur_pks5t8rU5gaJpZM4UoL3z .