Open batpad opened 2 months ago
@yuvipanda are you able to outline a little bit here what steps would be, and give me and @sunu some reading to get started?
And then we should probably get on a call early next week to kick this off.
Thanks!
yes let's do a call soon to kick this off!
Here's some prior reading that may be useful for @sunu!
I hope that's a useful start, @sunu!
@batpad how do we schedule this?
Thanks @yuvipanda! This is very useful. I'll give those threads a read.
Here's an update on the progress on this issue so far:
$ kubectl exec -it -n nfs-test nfs-client -- sh
/ #
/ # dd if=/dev/zero of=1mb.bin bs=1M count=1
1+0 records in
1+0 records out
1048576 bytes (1.0MB) copied, 0.002145 seconds, 466.2MB/s
/ # dd if=/dev/zero of=2mb.bin bs=1M count=2
2+0 records in
2+0 records out
2097152 bytes (2.0MB) copied, 0.002712 seconds, 737.5MB/s
/ # cp 1mb.bin /data/
/ # cp 2mb.bin /data/
cp: can't create '/data/2mb.bin': No space left on device
cc @batpad @yuvipanda
@sunu AMAZING! This looks great, continue on! I'd like more comments in https://github.com/sunu/jupyter-home-nfs/blob/main/nfs-ganesha/start.sh but otherwise awesome job!
Use https://github.com/jupyterhub/chartpress for CI and packaging. You can also git merge the get-quota-the-home repo into your repo (preserving history) to make this process easier too.
Excited to see this go!
Motivation
User home directories are meant to store code and notebooks, with data being stored in object storage if possible. They are mounted via NFS, and currently can grow unbounded. The unboundedness can cause cloud cost problems (unlike memory / CPU, which are bounded inherently), as well as encourage coding patterns that are not cloud optimized (like downloading a TB of .nc files to home directory before working on them).
Proposal
We will enforce home directory storage quotas for each user, as well as an overall fixed size for the entire hub. This will require migrating from the existing AWS EFS based setup to running an in-cluster NFS server with XFS based projectquota.
Owners
Success Criteria