Per-user dotfiles

[achimnol]

There are many customer requests to preserve the home directory contents, mainly user-installed packages.

The problem is, since our home directories are bind-mounted from each agent's scratch directory, they are not present in Docker images even we perform docker commit against running sessions. The technical reasons to use scratch directories are: 1) better I/O performance, 2) agent customization of home directory before starting containers, and 3) enforcement of quota limits.

Another problem is that some customer sites do not allow use of per-user vfolders due to their organizational policy, though we are offering auto-mounts of dot-prefixed per-user vfolders (e.g., .local and .ssh) if present to all sessions (#57) as an alternative to the above problem.

This issue tackles the second problem: per-user dotfiles.

For small configuration files aka dotfiles, we could just keep them in the user database and pass them to the agent whenever creating a new container, like SSH keypairs stored in the user database (lablup/backend.ai-manager#211). This will be exposed as "user settings" in the GUI and CLI.

• [x] manager: Add a blob column dotfiles (max-size limited to 64 KiB) which holds a msgpack'ed list of {"name": "<relative-path-and-file-name>", "perm": "644", "data": "<text-content>"} to the keypairs table.
• [x] manager: Add a new set of REST API to CRUD the dotfiles.
  • The total msgpack-encoded size of dotfiles must be less than 64 KiB and each file must be also smaller than 64 KiB. The total number of dotfiles is limited to 100.
  • Allow only valid UTF-8 encoded text files (not arbitrary binary data).
  • Prevent adding files under .local directory and the file .ssh/authorized_keys as they are managed by other means (#99 and lablup/backend.ai-manager#211).
• [x] manager/agent: Pass the value of dotfiles from manager to agent via internal_data field when creating new kernels.
• [x] agent: Populate the dotfiles when preparing the scratch directory.
• [ ] client: Add CLI & function interfaces for CRUD-ing dotfiles as backend.ai user-config dotfiles ... command set.

┆Issue is synchronized with this Asana task by Unito

[achimnol]

@kyujin-cho I missed to mention the CLI support. Please add the CLI support as well!

[achimnol]

Also related: lablup/backend.ai-manager#329