search

microsoft / vscode-remote-release

Visual Studio Code Remote Development: Open any folder in WSL, in a Docker container, or on a remote machine using SSH and take advantage of VS Code's full feature set.
https://aka.ms/vscode-remote
Other
3.55k stars 262 forks source link

Support Toolbox (a.k.a. Fedora Toolbox) #3345

Open lbssousa opened 3 years ago

lbssousa commented 3 years ago

Since Remote Containers extension actually supports Podman container runtime, I would like to go a step further and support container environments provided by Toolbox (do not confuse with Docker Toolbox).

Toolbox is a wrapper tool for Podman that allows creating and entering mutable containerized environments. I seems something like WSL approach, but for OCI containers via Podman.

There's already a way to connect VSCode to a Toolbox environment via Remote SSH extension, using this workaround, but I want to do the same directly via Remote Containers extension (or maybe a new Remote Toolbox one)

Maybe @debarshiray could provide more info about what's needed to add Toolbox support to VSCode Remote Development extensions.

chrmarti commented 3 years ago

Since it is a wrapper: Can the podman CLI be used to access it? If so, it should already be possible to connect to it with Remote-Containers.

DMouse10462 commented 3 years ago

Can the podman CLI be used to access it?

Yes, you can access it just like any regular container. I'm not the issue submitter, but I can share some of what I encounter when trying to connect to an existing toolbox.

Some basic info for my setup: OS: Fedora Workstation 32 $ podman -v: podman version 2.0.4

Logs from attempting to connect to a Fedora 32 toolbox named grails-dev:

[34 ms] Start: Resolving remote
[38 ms] Setting up container grails-dev

[39 ms] Start: Run: podman inspect --type container grails-dev
[297 ms] Start: Check Docker is running
[298 ms] Start: Run: podman info
[741 ms] Start: Inspecting container
[742 ms] Start: Run: podman inspect --type container ca9d153d0a5c2b4191be68bb3244b700416a87f5682b3db47b1f7f3a18dddcb7
[1190 ms] Start: Run: podman exec -i -u root -e VSCODE_REMOTE_CONTAINERS_SESSION=f54745c4-c0f0-4f9b-80be-0274471e9fbc1597217678921 ca9d153d0a5c2b4191be68bb3244b700416a87f5682b3db47b1f7f3a18dddcb7 /bin/sh
[1232 ms] Start: Run in container: uname -m
[1609 ms] x86_64
[1609 ms] 
[1610 ms] Start: Run in container: (cat /etc/os-release || cat /usr/lib/os-release) 2>/dev/null
[1614 ms] NAME=Fedora
VERSION="32 (Container Image)"
ID=fedora
VERSION_ID=32
VERSION_CODENAME=""
PLATFORM_ID="platform:f32"
PRETTY_NAME="Fedora 32 (Container Image)"
ANSI_COLOR="0;34"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:32"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f32/system-administrators-guide/"
SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=32
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=32
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
VARIANT="Container Image"
VARIANT_ID=container
[1614 ms] 
[1614 ms] Start: Run in container: cat /etc/passwd
[1620 ms] Start: Run in container: test -d /root/.vscode-server-insiders
[1624 ms] 
[1624 ms] 
[1625 ms] Start: Run in container: set -o noclobber ; mkdir -p '/root/.vscode-server-insiders/data/Machine' && { > '/root/.vscode-server-insiders/data/Machine/.writeMachineSettingsMarker' ; } 2> /dev/null
[1632 ms] 
[1632 ms] 
[1632 ms] Exit code 1
[1633 ms] Start: Run in container: test -d /root/.vscode-server-insiders/bin/2277c8e2a3e1cc630a6397301ba54a1dccd8a60d
[1639 ms] 
[1639 ms] 
[1639 ms] Start: Launching Remote-Containers helper.
[1642 ms] Start: Run: gpgconf --list-dir agent-extra-socket
[1659 ms] Start: Run in container: gpgconf --list-dir agent-socket
[1662 ms] /root/.gnupg/S.gpg-agent
[1663 ms] 
[1663 ms] Start: Run in container: gpgconf --list-dir homedir
[1667 ms] /root/.gnupg
[1667 ms] 
[1667 ms] Start: Run in container: ls '/root/.gnupg/private-keys-v1.d' 2>/dev/null
[1672 ms] 
[1672 ms] 
[1672 ms] Exit code 2
[1672 ms] Start: Run in container: mkdir -p -m 700 '/root/.gnupg'
[1680 ms] 
[1680 ms] 
[1680 ms] Start: Run in container: command -v git >/dev/null 2>&1 && git config --global credential.helper '!f() { /root/.vscode-server-insiders/bin/2277c8e2a3e1cc630a6397301ba54a1dccd8a60d/node /tmp/vscode-remote-containers-d8def71c0ee26c51418e0da1049888df62f6a191.js $*; }; f' || true
[1683 ms] Start: Run: gpgconf --list-dir homedir
[1699 ms] 
[1699 ms] 
[1700 ms] Start: Run in container: cat <<'EOF-/tmp/vscode-remote-containers-d8def71c0ee26c51418e0da1049888df62f6a191.js' >/tmp/vscode-remote-containers-d8def71c0ee26c51418e0da1049888df62f6a191.js
[1709 ms] 
[1709 ms] 
[1709 ms] Start: Run in container: set -o noclobber ; mkdir -p '/root/.vscode-server-insiders/data/Machine' && { > '/root/.vscode-server-insiders/data/Machine/.installExtensionsMarker' ; } 2> /dev/null
[1718 ms] 
[1721 ms] 
[1721 ms] Exit code 1
[1721 ms] Start: Run in container: cat <<'EOF-/tmp/vscode-remote-containers-server-d8def71c0ee26c51418e0da1049888df62f6a191.js' >/tmp/vscode-remote-containers-server-d8def71c0ee26c51418e0da1049888df62f6a191.js
[1767 ms] 
[1767 ms] 
[1767 ms] Start: Run in container: gpgconf --list-dir homedir
[1767 ms] Start: Run: podman exec -i -u root -e REMOTE_CONTAINERS_SOCKETS=["/tmp/vscode-ssh-auth-d8def71c0ee26c51418e0da1049888df62f6a191.sock","/root/.gnupg/S.gpg-agent"] -e REMOTE_CONTAINERS_IPC=/tmp/vscode-remote-containers-ipc-d8def71c0ee26c51418e0da1049888df62f6a191.sock ca9d153d0a5c2b4191be68bb3244b700416a87f5682b3db47b1f7f3a18dddcb7 /root/.vscode-server-insiders/bin/2277c8e2a3e1cc630a6397301ba54a1dccd8a60d/node /tmp/vscode-remote-containers-server-d8def71c0ee26c51418e0da1049888df62f6a191.js
[1797 ms] /root/.gnupg
[1797 ms] 
[1797 ms] Start: Run in container: for pid in `cd /proc && ls -d [0-9]*`; do { echo $pid ; readlink -f /proc/$pid/cwd ; xargs -0 < /proc/$pid/environ ; xargs -0 < /proc/$pid/cmdline ; } ; echo ; done 2>/dev/null
[1816 ms] Start: Run: gpg-connect-agent updatestartuptty /bye
[2301 ms] Remote-Containers server: Error: open executable: Permission denied: OCI runtime permission denied error
[2416 ms] Remote-Containers server terminated (code: 126, signal: null).
[3446 ms] Start: Run in container: /root/.vscode-server-insiders/bin/2277c8e2a3e1cc630a6397301ba54a1dccd8a60d/server.sh --disable-user-env-probe --use-host-proxy --port 0 --extensions-download-dir /root/.vscode-server-insiders/extensionsCache
[3458 ms] Error: stream is closed
        at /home/dmouse/.vscode-insiders/extensions/ms-vscode-remote.remote-containers-0.134.0/dist/extension.js:1:208142
        at c (/home/dmouse/.vscode-insiders/extensions/ms-vscode-remote.remote-containers-0.134.0/dist/extension.js:1:224558)
        at EventEmitter.e.<computed> [as exec] (/home/dmouse/.vscode-insiders/extensions/ms-vscode-remote.remote-containers-0.134.0/dist/extension.js:1:224281)
        at u (/home/dmouse/.vscode-insiders/extensions/ms-vscode-remote.remote-containers-0.134.0/dist/extension.js:1:202138)
        at new Promise (<anonymous>)
        at /home/dmouse/.vscode-insiders/extensions/ms-vscode-remote.remote-containers-0.134.0/dist/extension.js:1:202116
        at J (/home/dmouse/.vscode-insiders/extensions/ms-vscode-remote.remote-containers-0.134.0/dist/extension.js:1:81982)
        at W (/home/dmouse/.vscode-insiders/extensions/ms-vscode-remote.remote-containers-0.134.0/dist/extension.js:1:81174)
        at processTicksAndRejections (internal/process/task_queues.js:85:5)
        at async Object.t.injectIntoContainer (/home/dmouse/.vscode-insiders/extensions/ms-vscode-remote.remote-containers-0.134.0/dist/extension.js:1:77410)
        at async Object.t.resolve (/home/dmouse/.vscode-insiders/extensions/ms-vscode-remote.remote-containers-0.134.0/dist/extension.js:1:238639)
        at async /home/dmouse/.vscode-insiders/extensions/ms-vscode-remote.remote-containers-0.134.0/dist/extension.js:1:688588

When I manually go into the toolbox and attempt to run the server.sh command (as root within the container due to permission issues; the toolbox command usually drops you to a UID 1000 shell with sudo privileges), the server appears to start correctly unlike the output within VS code. Here's the output of me running the command within a toolbox enter shell (Addendum: the same results occur when entering via podman exec and running the command directly as root):

⬢[dmouse@toolbox ~]$ sudo /root/.vscode-server-insiders/bin/2277c8e2a3e1cc630a6397301ba54a1dccd8a60d/server.sh --disable-user-env-probe --use-host-proxy --port 0 --extensions-download-dir /root/.vscode-server-insiders/extensionsCache

*
* Visual Studio Code Server
*
* Reminder: You may only use this software with Visual Studio family products,
* as described in the license https://aka.ms/vscode-remote/license
*

IP Address: <MY HOST IP>
Extension host agent listening on 34411

[03:44:20] Extension host agent started.
Unable to retrieve mac address (Error: Command failed: /sbin/ifconfig -a || /sbin/ip link
/bin/sh: /sbin/ifconfig: No such file or directory
/bin/sh: /sbin/ip: No such file or directory
)
^C
⬢[dmouse@toolbox ~]$ id
uid=1000(dmouse) gid=1000(dmouse) groups=1000(dmouse),10(wheel)

It is worthy of note that the IP address shown is the address of the host despite running within a container; the user's home directory is mounted in a toolbox and the container appears to be running in host networking mode (?). I'm not 100% sure of what else toolbox sets up the container with.

lbssousa commented 3 years ago

@debarshiray, could you give us any useful info about Toolbox here?

debarshiray commented 3 years ago

Very excited to see the interest in Toolbox here!

Toolbox containers are just OCI containers created, managed and run by Podman under the hood. Toolbox is a thin wrapper around Podman to set up the containers in a way that makes it easy to use them as pet development environments without being burdened by the cognitive load of being inside a container.

Assuming that you have already created a container (with toolbox create ...), entering or using the container is a matter of:

I hope that this gives you some idea about what happens underneath the toolbox enter command.

I am happy to answer more specific questions.

likan999 commented 3 years ago

I have a PR for toolbox to make it work: https://github.com/containers/toolbox/pull/587. With the patch, on your local machine, you only need an alias in ~/.ssh/config that has RemoteCommand toolbox run -T bash and everything just works as a normal SSH remote host. Of course VScode could do fancier stuff (e.g. without need to manually modify the ssh config) but at least it is no longer blocked on the toolbox side.

likan999 commented 3 years ago

To clarify, my approach is different to the workaround mentioned in the top. The workaround starts an openssh server inside the toolbox, while my approach doesn't rely on any service running inside the toolbox.

lbssousa commented 3 years ago 
* Calling the `org.freedesktop.Flatpak.SessionHelper.RequestSession` D-Bus method

* Starting the container with `podman start ...` and waiting for the entry-point to have performed some run-time configuration

* Entering it with `podman exec ...`

@chrmarti , can these steps (specially the first one) be performed with current Remote Containers implementation, or there's some lacking piece?

chrmarti commented 3 years ago

@lbssousa There is "initializeCommand" in the devcontainer.json that runs before anything else. We support Podman with a setting for the Docker executable that can be set to podman. We then start the container with podman run and connect to it with podman exec. Not sure what the entry-point needs to do, but you might need to turn off our overriding of it with "overrideCommand": false in the devcontainer.json (and then make sure the container still keeps running).

lbssousa commented 3 years ago

@chrmarti, I was able to attach to my Toolbox container with the following minimal devcontainer.json:

{
  "name": "Toolbox",
  "image": "registry.fedoraproject.org/f33/fedora-toolbox:33",
  "initializeCommand": "gdbus call --session --dest org.freedesktop.Flatpak --object-path /org/freedesktop/Flatpak/SessionHelper --method org.freedesktop.Flatpak.SessionHelper.RequestSession"
  "overrideCommand": false // I see no difference in setting this option or not
}

However, VSCode always connect to that container as root (remoteUser does not work for me, since it seems my host user is not actually created inside the container: toolbox enter still does some magic to make my host user available to the container).

@debarshiray, is there something else I need to do? What shoud I do with the output of gdbus call ... org.freedesktop.Flatpak.SessionHelper.RequestSession command?

EDIT: My mistake. With the approach above, I'm not attaching to the same container created by toolbox create, but, instead, creating a new container from the same image. When I try to attach to the correct Toolbox container, I get the same error reported earlier:

Remote-Containers server: Error: open executable: Permission denied: OCI runtime permission denied error
debarshiray commented 3 years ago

What shoud I do with the output of gdbus call ... org.freedesktop.Flatpak.SessionHelper.RequestSession command?

Note that this isn't necessary any more for Toolbox containers created with current toolbox.git, which will be released as Toolbox 0.0.97.

It's possible to detect whether it's necessary or not by running podman inspect on the container and checking if it bind mounts anything at /run/host/monitor.

The overall idea was to somehow keep the time zone configuration of the container synchronized with the host. So far, Toolbox made use of the flatpak-session-helper utility to keep a copy of the host's configuration files in $XDG_RUNTIME_DIR/.flatpak-helper/. This location was then bind mounted inside the container at /run/host/monitor, and symbolic links were set up inside the container's /etc.

Most of this is done as part of the podman create ... call that creates the Toolbox container, and the entry point of the container itself. However something needs to launch the flatpak-session-helper before starting the container so that things are available in $XDG_RUNTIME_DIR/.flatpak-helper/, and this is what that gdbus call ... command does.

debarshiray commented 3 years ago

Remote-Containers server: Error: open executable: Permission denied: OCI runtime permission denied error

This could be Podman territory, but we need some more information to be sure. Which is the exact command that's failing?

I don't know very well how vscode-remote-release works, so I will throw some more information out there. :)

All Toolbox containers have their entry points set to toolbox init-container .... This is important because it's used to self-configure the container at run-time. Yes, /usr/bin/toolbox is also the entry point. This is achieved by bind mounting /usr/bin/toolbox from the host into the container and then using it.

/usr/bin/toolbox enter uses podman start ... to start the container and then podman exec ... to enter it. If you are instead using podman run ... to start the container, then that might be running into some problems. It might be possible to navigate them, but it might also be better to just use podman start ... if that's possible.

Note that the podman exec ... invocation also takes a long list of arguments that includes environment variables and what not.

Would it somehow be possible to just use toolbox enter to enter the container?

chrmarti commented 3 years ago

We are trying to stay out of adaptions specific to one CLI or runtime. Podman-support comes from Podman's CLI being compatible with the Docker CLI, so we didn't have to change much (except for the version check).

lbssousa commented 3 years ago

@debarshiray , I'll keep investigating the Remote-Containers's approach here. For now, it seems @likan999 's PR approach is the easiest one.

lbssousa commented 3 years ago

I finally realized that, although Toolbox environment are simple OCI containers instantiated by Podman under the hood, they are not expected to be handled the same way a typical application container is, but, instead, more like a WSL distro. Thus, VSCode's Remote-Containers extension may not be the best way to attach to Toolbox containers - we may need another Remote-Toolbox one, similar to Remote-WSL. I'm not sure, however, if such an extension should be developed by Microsoft itself, or if can be developed by the community, since VSCode's Remote Development infrastructure seems to be closed-source. @chrmarti

In the case such an extension to be created, it should be interesting, in Toolbox side, if folder ~/.vscode-server could be bound to an internal volume, so it wouldn't be exposed to host system's user home folder. @debarshiray

debarshiray commented 3 years ago

I finally realized that, although Toolbox environment are simple OCI containers instantiated by Podman under the hood, they are not expected to be handled the same way a typical application container is, but, instead, more like a WSL distro.

Indeed. Toolbox containers are meant to be long-lived so-called pet containers.

In the case such an extension to be created, it should be interesting, in Toolbox side, if folder ~/.vscode-server could be bound to an internal volume, so it wouldn't be exposed to host system's user home folder.

You mean some location on the host OS that's outside $HOME should be bind mounted at ~/.vscode-server inside the container? Sounds doable. What kind of host location do you have in mind?

HarryMichal commented 3 years ago

Hi! I'm working on Toolbox together with @debarshiray. I just spent a little while debugging this and came up with a few points:

  1. The Toolbox usecase seems to only apply to Remote-Containers: Attach to a running container, therefore there's no need to worry about creating nor starting the container (the magic behind Toolbox is not in the images but in the way the containers are created using toolbox create)
  2. Running the extension in VSCode flatpak is tough but it's doable (mentioning this to give info about my dev env)
  3. As already described, the connection to a toolbox fails with: 
    Remote-Containers server: Error: open executable: Permission denied: OCI runtime permission denied error

    I managed to reproduce this error with the following command:

    podman exec -u root golang /root/.vscode-server/bin/d2e414d9e4239a252d1ab117bd7067f125afd80a/server.sh --disable-user-env-probe --use-host-proxy --disable-telemetry --port 0 --extensions-download-dir /root/.vscode-server/extensionsCache

    The golang container is a toolbox created with toolbox create golang.

While playing with it, I discovered that the problem can be solved by passing the --tty option to podman exec (but I recommend also passing the -i option to prevent your terminal session getting stuck). This is something that also GNOME Builder does in its Podman plugin.

lbssousa commented 3 years ago

@HarryMichal @chrmarti it seems not to be just a matter of attaching to a running (Toolbox) container. It may be needed to pass additional command-line arguments when attaching to that container. I'm not sure if VSCode's Remote-Containers extension currently supports such a feature (e.g. attach-only devcontainer.json file).

HarryMichal commented 3 years ago

I'm no expert when it comes to the Remote-Containers extensions as I never got it to work on my system. My use process was based on (from my point of view) the expected workflow:

  1. Click the button in the left bottom corner
  2. Choose an option you think fits your needs the most
  3. Follow the instructions

And yes, it seems it is needed to extend the extension code for the options I mentioned to get toolboxes running. That is completely up to the devs if they see fit to add them, unless somebody else comes up with a different solution (plus maybe a different cause of the problem).

chrmarti commented 3 years ago

I don't see why -t and -i would affect the permissions error. Also: This seems to work with other Podman containers. Maybe I'm missing something?

lbssousa commented 3 years ago

@chrmarti is it currently possible to attach to a running container from its container name (not previously built from a Dockerfile or docker-compose.yml) as a non-root user, like we do for application containers setting remoteUser property in devcontainer.json? I think this may be needed in order to attach properly to a Toolbox pet container.

Remote-Containers' documentation says:

select Remote-Containers: Open Named Configuration File... from the Command Palette

but I'm unable to find this option. Instead, I'm only finding Remote-Containers: Open Attached Container Configuration File...

lbssousa commented 3 years ago

@chrmarti @Chuxel @egamma To be more specific: toolbox run -c my-toolbox [command...] calls, under the hood, a command like this:

podman --log-level error exec \
    --detach-keys= \
    --interactive \
    --tty \
    --user ${USER} \
    --workdir ${HOME} \
    --env=COLORTERM=${COLORTERM} \
    --env=DBUS_SESSION_BUS_ADDRESS=${DBUS_SESSION_BUS_ADDRESS} \
    --env=DESKTOP_SESSION=${DESKTOP_SESSION} \
    --env=DISPLAY=${DISPLAY} \
    --env=LANG=${LANG} \
    --env=SHELL=${SHELL} \
    --env=SSH_AUTH_SOCK=${SSH_AUTH_SOCK} \
    --env=TERM=${TERM} \
    --env=VTE_VERSION=${VTE_VERSION} \
    --env=XDG_CURRENT_DESKTOP=${XDG_CURRENT_DESKTOP} \
    --env=XDG_DATA_DIRS=${XDG_DATA_DIRS} \
    --env=XDG_MENU_PREFIX=${XDG_MENU_PREFIX} \
    --env=XDG_RUNTIME_DIR=${XDG_RUNTIME_DIR} \
    --env=XDG_SESSION_DESKTOP=${XDG_SESSION_DESKTOP} \
    --env=XDG_SESSION_TYPE=${XDG_SESSION_TYPE} \
    --env=TOOLBOX_PATH=/usr/bin/toolbox \
    my-toolbox \
    capsh \
    --caps= \
    -- \
    -c 'exec "$@"' \
    /bin/sh [command ...]

Note: toolbox enter is nearly the same as toolbox run ${SHELL} -l

Is it possible to reproduce such command-line arguments with Remote-Containers extension when attaching to a running container?

@debarshiray When I try to mimic toolbox run by running the command above explictly in bash prompt after starting my-toolbox container, I've got the following error:

~~WARN[0000] Failed do add conmon to systemd sandbox cgroup: Unit libpod-conmon-XXXXXXXXXXXXX.scope already exists. Error: container create failed (no logs from conmon): EOF~~

Is there something else I should do to prevent this?

Nevermind, it was just an empty string missing as argument to option --detach-keys.

chrmarti commented 3 years ago

You get Remote-Containers: Open Named Configuration File... only while already attached to a container. "remoteUser" also works with attach configs.

The files are in $HOME/.config/Code/User/globalStorage/ms-vscode-remote.remote-containers/nameConfigs/${container name}.json (.config/Code\ -\ Insiders/... for VS Code Insiders).

We user docker/podman exec for attaching to an already running container. Are you asking if that can be changed? Currently not.

lbssousa commented 3 years ago

So, in order to Remote-Containers to mimic toolbox enter properly, it should support a file $HOME/.config/Code/User/globalStorage/ms-vscode-remote.remote-containers/nameConfigs/${TOOLBOX_CONTAINER_NAME}.json like this:

{
  "remoteUser": "${localEnv:USER}",
  "workspaceFolder": "${localEnv:HOME}",
  "remoteEnv": {
    "TOOLBOX_PATH": "/usr/bin/toolbox",
    "COLORTERM": "${localEnv:COLORTERM}",
    "DBUS_SESSION_BUS_ADDRESS": "${localEnv:DBUS_SESSION_BUS_ADDRESS}",
    "DESKTOP_SESSION": "${localEnv:DESKTOP_SESSION}",
    "DISPLAY": "${localEnv:DISPLAY}",
    "LANG": "${localEnv:LANG}",
    "SHELL": "${localEnv:SHELL}",
    "SSH_AUTH_SOCK": "${localEnv:SSH_AUTH_SOCK}",
    "TERM": "${localEnv:TERM}",
    "VTE_VERSION": "${localEnv:VTE_VERSION}",
    "XDG_CURRENT_DESKTOP": "${localEnv:XDG_CURRENT_DESKTOP}",
    "XDG_DATA_DIRS": "${localEnv:XDG_DATA_DIRS}",
    "XDG_MENU_PREFIX": "${localEnv:XDG_MENU_PREFIX}",
    "XDG_RUNTIME_DIR": "${localEnv:XDG_RUNTIME_DIR}",
    "XDG_SESSION_DESKTOP": "${localEnv:XDG_SESSION_DESKTOP}",
    "XDG_SESSION_TYPE": "${localEnv:XDG_SESSION_TYPE}",
  },
  "execArgs": [
    "capsh",
    "--caps=",
    "--",
    "-c 'exec \"$@\"'",
    "/bin/sh",
    "${localEnv:SHELL}",
    "-l"
  ]
}

(UPDATE 2020-11-12: There's no need to set TOOLBOX_PATH here, because it's already defined in the container's environment. There's also no need to set workspaceFolder, since it already goes to my home folder by default after attaching)

Moreover, it should spawn podman exec command with flags --interactive and --tty.

However, it seems neither ${localEnv:ENV_VAR} nor execArgs are currently supported for attached containers configuration. execArgs support is mandatory here, but the lack of ${localEnv:ENV_VAR} support could be circumvented if toolbox provides a way to export a suitable $HOME/.config/Code/User/globalStorage/ms-vscode-remote.remote-containers/nameConfigs/${TOOLBOX_CONTAINER_NAME}.json with those values properly set. @debarshiray (UPDATE 2020-11-12): ${localEnv:VAR} are supported in attached containers configuration for both remoteUser and remoteEnv properties.

When I try to attach to my Toolbox container with such a configuration (setting explicitly my username in remoteUser and my home dir in workspaceFolder), it ends up with the following error:

[2 ms] Remote-Containers 0.148.1 in VS Code 1.51.0 (fcac248b077b55bae4ba5bab613fd6e9156c2f0c).
[11 ms] Start: Resolving remote
[14 ms] Setting up container fedora-toolbox-33

[15 ms] Start: Run: podman inspect --type container fedora-toolbox-33
[83 ms] Start: Check Docker is running
[84 ms] Start: Run: podman info
[180 ms] Start: Inspecting container
[180 ms] Start: Run: podman inspect --type container 9c9e9954f7a86b2b181eeb3fa98bc1f8151d30adf3979577e9e08b954dc35358
[236 ms] Start: Run: podman exec -i -u laercio -e VSCODE_REMOTE_CONTAINERS_SESSION=7be72b4f-5d91-48a7-9f0a-c4cd512302ef1605100356707 9c9e9954f7a86b2b181eeb3fa98bc1f8151d30adf3979577e9e08b954dc35358 /bin/sh
[239 ms] Start: Run in container: uname -m
[325 ms] x86_64
[326 ms] 
[326 ms] Start: Run in container: (cat /etc/os-release || cat /usr/lib/os-release) 2>/dev/null
[330 ms] NAME=Fedora
VERSION="33 (Container Image)"
ID=fedora
VERSION_ID=33
VERSION_CODENAME=""
PLATFORM_ID="platform:f33"
PRETTY_NAME="Fedora 33 (Container Image)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:33"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f33/system-administrators-guide/"
SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=33
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=33
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
VARIANT="Container Image"
VARIANT_ID=container
[330 ms] 
[331 ms] Start: Run in container: cat /etc/passwd
[333 ms] Start: Run in container: test -d /root/.vscode-server
[334 ms] 
[334 ms] 
[334 ms] Exit code 1
[334 ms] Start: Run in container: test -d /root/.vscode-remote
[335 ms] 
[335 ms] 
[335 ms] Exit code 1
[335 ms] Start: Run in container: set -o noclobber ; mkdir -p '/root/.vscode-server/data/Machine' && { > '/root/.vscode-server/data/Machine/.writeMachineSettingsMarker' ; } 2> /dev/null
[337 ms] 
[337 ms] mkdir: cannot create directory '/root': Permission denied
[337 ms] Exit code 1
[338 ms] Start: Run in container: test -d /root/.vscode-server/bin/fcac248b077b55bae4ba5bab613fd6e9156c2f0c
[339 ms] 
[339 ms] 
[339 ms] Exit code 1
[339 ms] Installing VS Code Server for commit fcac248b077b55bae4ba5bab613fd6e9156c2f0c
[339 ms] Start: Run in container: mkdir -p /root/.vscode-server/bin/fcac248b077b55bae4ba5bab613fd6e9156c2f0c_1605100357608
[345 ms] 
[346 ms] mkdir: cannot create directory '/root': Permission denied
[346 ms] Exit code 1
[349 ms] Command in container failed: mkdir -p /root/.vscode-server/bin/fcac248b077b55bae4ba5bab613fd6e9156c2f0c_1605100357608

The problem here is that Remote-Containers is trying to install Remote VSCode server at /root/.vscode-server rather than $HOME/.vscode-server. Is there a way to make it work?

chrmarti commented 3 years ago

It looks up HOME first on the container's environment variables (using inspect) and then in /etc/passwd for that user. One of the two must be /root here, try updating that.

${localEnv:VAR} also works for attach configs, maybe HOME and/or USER are not set for the VS Code processes?

We don't have execArgs for any of our configs currently.

lbssousa commented 3 years ago

In fact, after running podman inspect ${TOOLBOX_CONTAINER_NAME}, I've found HOME=/root in its environment. After giving write permission to container's /root folder for my user, I was finally able to attach to my Toolbox container.

I've verified that ${localEnv:VAR} is working properly - in fact, even "remoteUser": "${localEnv:USER}" works. It only fails if I try to set "workspaceFolder": "${localEnv:HOME}", but I just realized that it's not needed at all - after removing that line from my config, Remote-Containers goes to my home folder by default after attaching to container.

I was still unable to pass arguments capsh --caps= -- -c 'exec "$@"' /bin/sh [my-shell] to podman exec. Nevertheless, I've got correctly my shell prompt after attaching to my Toolbox container. @debarshiray , is this really needed?

In short, there seems some missing pieces in Remote-Containers extension to make my life easier when attaching to a Toolbox container:

chrmarti commented 3 years ago

You could unset HOME with:

    "containerEnv": {
        "HOME": ""
    }

in the devcontainer.json.

lbssousa commented 3 years ago

Unsetting HOME in containerEnv had no effect here: it still tries to install .vscode-server under /root. VSCode's documentation says the container needs to be rebuilt for this to take effect, which is out of question for my case.

About those capsh ... arguments, I've found a workaround. I've just added these settings to attached container config:

{
  "settings": {
    "terminal.integrated.shell.linux": "/usr/sbin/capsh",
    "terminal.integrated.shellArgs.linux": [
      "--caps=",
      "--",
      "-c",
      "exec \"$@\"",
      "/bin/sh",
      "${localEnv:SHELL}",
      "-l"
    ]
  }
}

One more suggestion to make my life easier is allowing access to command Remote-Containers: Open Named Configuration File... in command palette prior to attaching to container, or just providing a similar command that allows me to create an attach config before attaching to container.

chrmarti commented 3 years ago

Did you rebuild the container after changing containerEnv? I had unsetting working, try setting it to the desired path if it doesn't.

lbssousa commented 3 years ago

No, I didn't rebuilt my container after setting "containerEnv". I tried both empty string and my home folder path, but none of tem has worked.

TBH, I'm not worried about this anymore: keeping .vscode-server under /root folder within the container allows me not leaking it to my host system's home folder.

lbssousa commented 3 years ago

There's one last thing that still makes me unconfortable with this solution: since Toolbox is a very special case of container-based development, in which host system's user home folder is directly bound to the container, Remote-Containers extension ends up overwriting my ~/.gitconfig's credential.helper value, replacing the original value (e.g. store) with something like "!f() { /root/.vscode-server/bin/e5a624b788d92b8d34d1392e4c4d9789406efe8f/node /tmp/vscode-remote-containers-0f1183fad6e028a06ad0d6f114ae48cb785a1d17.js $*; }; f". Is there a way to prevent this?

chrmarti commented 3 years ago

There is a Remote-Containers setting to turn this off.

lbssousa commented 3 years ago

Oh, I've just found it here. Thanks!

For anyone interested, I've summarized my configuration to make Remote-Containers work with my Toolbox container here.

debarshiray commented 3 years ago

It's really amazing that you have managed to get this far!

The part about HOME being set to /root makes me curious. A Toolbox container uses two UIDs. One is UID 0. This isn't the actual UID 0 on the host, but some other UID on the host mapped to 0 inside the container's user namespace. This is used to run the container's entry point. I believe this is why you see the HOME=/root. The other is the user's own UID which is the same both on the host and in the container. This is what is used to present the shell to the user.

Is Visual Studio Code going by the HOME it sees in the podman inspect output when trying to place the .vscode-server directory?

chrmarti commented 3 years ago

@debarshiray Yes, we first look at HOME from podman inspect. If that is not set we look up it in /etc/passwd. Note that we install and run the server as the remoteUser which might run into permission issues under /root.

lbssousa commented 3 years ago

Does Remote-Containers extension currently support property name for attached container config? I'm trying to customize my Toolbox display name in VSCode, but it's not working.

If not, could you amend it to issue #4013?

lbssousa commented 3 years ago

I guess we have now all the needed information for an eventual solution on the Toolbox side (auto-generating the needed configuration for Remote-Containers). I've opened an issue there. If @debarshiray has no other questions, I believe this issue can be closed now.

alvarlagerlof commented 2 years ago

Sorry for the ping, but I'm seeing a lot of issues here and I'm having trouble making sense of what works and what doesn't. Is there an updated guide/text on what works and how to set that up?

lbssousa commented 2 years ago

@alvarlagerlof The most up-to-date solution about this issue is a third-party project: https://github.com/owtaylor/toolbox-vscode

alvarlagerlof commented 2 years ago

@alvarlagerlof The most up-to-date solution about this issue is a third-party project: https://github.com/owtaylor/toolbox-vscode

That does not work anymore. Any alternatives?

heyakyra commented 2 years ago

This might finally get me to switch to ~kakoune~ Helix / Lapce with zellij + tmate

daw1012345 commented 1 year ago

Almost 3 years after this issue was originally created I ran into problems trying to use Toolbox containers and VScode together. https://github.com/owtaylor/toolbox-vscode doesn't seem to be working and other articles on the topic caused other (permission) issues for me. I decided to write an updated script that works well enough for me.