Proposal(lang-python): Use fuse-overlayfs instead of pyenv-hooks for persistence.

[axonasif]

Using pyenv-hook for setting up python/pip persistence over pyenv bought in unwanted issues and increased a maintenance burden, something that I had not anticipated (https://github.com/gitpod-io/workspace-images/pull/868).

Before going ahead with pyenv-hooks I had tried native kernel overlayfs but that didn't work^1. Recently, I looked for alternative methods in an effort to simplify the pyenv setup, it seems that a fuse implementation of overlayfs works.

By using fuse-overlayfs, we can get rid of all the custom logic that was implemented via pyenv-hook bash scripts.

with pyenv-hooks	with fuse-overlayfs

Implications:

• Requires the host workspace to have privileged access to /dev/fuse, if not found it will simply skip mounting on $HOME/.pyenv and things will keep working normally except they will lack persistence between prebuilds/reboots.
• Unsure if it got any drawbacks.

Questions

• Does it work inside gp validate environment?
  • Yes.
• When is it mounted?
  • It is mounted when $GITPOD_REPO_ROOT exists. Means it will be mounted before supervisor tasks are executed in a prebuild-workspace or user-workspace. Uses an atomic lock to avoid race conditions and makes all the terminals await for the completion.

Proposed implementation

• Remove pyenv-hooks.
• Remove run-time variables and define them statically from the Dockerfile.
• Install fuse-overlayfs in the base image.
• Use a helper script (similar to install-packages) that wraps around fuse-overlayfs for easy use.

You can check the following commit for an example implementation: https://github.com/axonasif/workspace-images/commit/d9ec66af5abcae2ec9e1381e62aa7f9d9d150670

A docker image with the above change is available as: axonasif/workspace-python:debug2

• Testing from here: https://github.com/axonasif/test/tree/python_fix_test

fuse-overlayfs for solving Ruby issues

It could help with Ruby and nix as well with one simple command.

Some recent examples

- https://discord.com/channels/816244985187008514/1082108211936645182 ---- - https://app.frontapp.com/open/msg_yjlruq2?key=Pdup57WdyslSRCq3sstVc3uX6wANTOQN

[axonasif]

Here's another proposed solution based on the discussion I had with @akosyakov on here.

Uses python -m venv <dir> (for python 3.3+) or virtualenv command to create an env inside /workspace on runtime and activate the env.

At first I was trying to use pyenv-virtualenv plugin of pyenv, but it creates the envs inside $HOME/.pyenv/versions/<dir>. However the plugin is essentially a wrapper around python -m venv/virtualenv, so we can use them to create envs directly inside /workspace without the pyenv plugin.

Commit: https://github.com/axonasif/workspace-images/commit/480fadd7306fb2f6cf44ac9ba03f3bfcdbe78480

Experimental image: axonasif/workspace-python:debug7

Try on a repo: https://github.com/axonasif/test/tree/python_fix_test_venv

Implications

• Python libraries installed from a custom dockerfile will not be available during runtime, this includes our own lang-python/Dockerfile Update: Workaround: https://github.com/axonasif/workspace-images/commit/eba48e6df572049f1fcd113ff5de4520414a646c
• Unsure if it got any drawbacks.