Adjust home path - Githubissues

ckaspro commented 4 years ago

I like to use several toolboxes, with different content. To properly distinguish these, I'd like to mount them in subdirectories (e.g. container1:/home/user -> host:/var/home/user/container1). It's a similar thing to #183 , but not as security- centric. Is there a way to achieve this at the moment / via consistent workaround (I don't want to change a line of code and after the next update I'm saving files somewhere else ;) )?

Update: I wouldn't actually mind (maybe even appreciate) having these under different user-contexts - but in the same general directory. That way there is a clear distinction and separation. For presenting this to the end- user in a "nice way", one could add an extra link opening the file-browser as sudo... (just thinking loud here)

Thanks in advance, Chris

yilkalargaw commented 4 years ago

I usually just do HOME=/home/user1/container1 before creating the containers and just use an alias to open them. (alias toolbox1='HOME=/home/user1/container1/toolbox "$@"')

debarshiray commented 4 years ago

Yeah, overriding $HOME is one way to do it.

JaneSmith commented 4 years ago

I'd also like to see something like this. My main use case for Toolbox is to set up throwaway development environments, for either working on projects, or just quickly compiling some software from source. I don't want to pollute my system with random files from dependencies, etc. just to compile something once. Toolbox seemed perfect for this, as it claims to provide an isolated container to keep the host clean.

This didn't turn out to work quite how I expected though. While Toolbox does keep the host OS clean, it doesn't keep the home directory clean at all. That still gets totally polluted. For example, using a throwaway container to build something that uses rust, my home directory ended up getting altered in several ways:

A ~/.cargo directory was silently created, containing many Rust-related binaries, source packages, etc. for a total of 123 MB.
A ~/.rustup directory was silently created, containing many more Rust-related binaries, for a total of 683 MB.
My ~/.bash_profile file was silently altered to add the ~/.cargo/bin directory to my $PATH ahead of everything else.
My ~/.profile file was silently altered to add the ~/.cargo/bin directory to my $PATH ahead of everything else.
Who knows what else.

Yikes. What was supposed to be a temporary, throwaway, easy-to-discard system resulted in many permanent changes being silently made to my home directory. I realise now from the comments above that I can override the $HOME environment variable manually to try to work around this, but this was not intuitive or expected to me. Since Toolbox is supposed to be (or at least as I understand it) a simplified and user-friendly way to get into containers, I would appreciate if this could be handled in some better way.

My opinion is that a Toolbox should probably have its own unique user and home directory by default. But if this is too complicated to implement, then perhaps there could at least be a -h or --home argument when creating a new toolbox, to set its default $HOME? So that when you enter the toolbox in future, it would get that $HOME value set automatically. For example, something along the lines of toolbox create -c temp-myproject -h ~/Toolboxes/temp-myproject.

Basically, I think it would be great if Toolbox had a user-friendly way to set up a container in one of three modes:

Seamless mode: How it works now. The container user acts as if it's your real host user, sharing your home directory.
Semi-isolated mode: The container has access to your host user's files, but has its own home directory for software to use by default. You would need to manually access your host user's home directory if you want to read/write to there. It would basically be like the Seamless mode above, but with its own separate working directory.
Fully-isolated (untrusted) mode: The container would have its own separate user and home directory, with absolutely no access to your host user's home directory. This would be useful for running untrusted software, which you don't want to allow to read everything in your home directory.

HarryMichal commented 4 years ago

This sounds very reasonable to me. What do you think @debarshiray??

ckaspro commented 4 years ago

I support this @JaneSmith , all I need - nicely put!

yilkalargaw commented 4 years ago

I second the idea too. But until it gets implemented you could use direnv. It is a handy tool that allows one to specify environment variables on a per directory basis. You can use the following work flow

- install direnv and hook it to your shell
- create a temporary directory eg. ~/somedev
- add a .envrc file with the environment variables you want (in this case HOME=/home/user/somedev

now every time you go into the somedev directory it will assume it is the home directory and resets when you step out of it . You could do all the steps after installing direnv in one command like:

mkdir ~/somedev && echo export HOME=/home/user/somedev > ~/somedev/.envrc

I currently use a similar work flow.

rugk commented 4 years ago

I agree there should at least be an option to not mount the home dir (ref https://github.com/containers/toolbox/issues/183#issuecomment-623103780). Actually, I would even be fine if it uses – similar to flatpak – some dir in ~/.var/app, maybe better ~/.var/toolbox or so, where all files in the home dir are saved by default. I quite like that flatpak model, because you have all the data from one container in one place, and if you delete the container, you can also delete all app data. (or just measure the space your new "I've tried out rust" project took), e.g. (as it's done in gnome-control-center)

Then, for convenience, it should possibly always mount in the current $PWD, if you start it from a project folder, you likely expect to have the files in there. Just, your ~/rust-project folder, does not need any access to ~/Pictures/private/… etc.

Anyway, of course one should be able to optionally mount home as read-only or even writable, but I don't think that is really needed for most applications. And one should be able to not mount the current $PWD.

So you'd have a "middle-ground" here as a new default.

markstos commented 4 years ago

tlbx fork has a -n option to not bind-mount the home directory.

debarshiray commented 4 years ago

Duplicate of https://github.com/containers/toolbox/issues/183

rugk commented 4 years ago

Noooo… you can still have other use cases/reasons for having a different $HOME then an "Isolated development environments to fend against bugs and malware in code under development"…

IMHO, this is still a feature that toolbox should have.

JaneSmith commented 4 years ago

I don't understand why this issue was closed. It's not a duplicate of that. This isn't just about security. I really hope this will be reconsidered as I love Fedora Silverblue and love that it has a built-in tool to set up pet containers, but it doesn't really do the job currently. I don't want to have to use a fork just for that... I use toolbox for pet containers to deal with various programming projects, and I don't want my home directory to be cluttered by the containers. It has nothing to do with security.

markstos commented 4 years ago

@JaneSmith why not use a fork if it has the features you need?

JaneSmith commented 4 years ago

I'd prefer to use Toolbox rather than a fork, because Toolbox is better supported with more developers, and is included with Fedora out of the box. When I'm moving from machine to machine, Toolbox is already there, without me having to install forks. Part of the reason for using Toolbox in the first place is to avoid cluttering my system with random software installations!

In my humble opinion, this feature request is for something that should be core, basic functionality, as not having it kind of defeats the purpose of Toolbox. I don't see it as a really "out there" unusual special feature. I'm therefore voicing my support for it and hoping that it will be implemented for this project.

markstos commented 4 years ago

@JaneSmith I agree on all points. I expect the feature will be implemented the first time the current insecure design is publicized to play a part in a security breach, if not sooner.

markstos commented 4 years ago

The first sentence in the docs says it runs fully unprivileged . Sounds secure, huh?

Also in the main description: The intention of these systems is to discourage installation of software on the host, and instead install software as (or in) containers. Also sounds like this project must add significant security.

Nowhere do the docs highlight that it's sharing every document your user owns with with every container, including SSH keys and files unrelated to what's happening in the container.

The current situation is not only insecure, but misleads to convey that using containers like this provides significant security, when it's hardly the case the containers can access all your files!

If you search for "home" in the README, there is no documentation that your home directory is shared in this way.

ChrisTucker2000 commented 4 years ago

I want this too. A middle ground (home accessible, but not defined as HOME), and a more untrusted mode (home not accessible at all).

I usually just do HOME=/home/user1/container1 before creating the containers and just use an alias to open them. (alias toolbox1='HOME=/home/user1/container1/toolbox "$@"')

If a middle ground solution is this simple... then why can't it be implemented into Toolbox as an option? Just take the home path as an argument when creating the toolbox and save it somewhere, probably the same way as how the container name is saved. Then read that path and set it when entering the container...

swick commented 3 years ago

To make things a bit more concrete:

--new-home which would tell toolbox enter to not mount host $HOME (or mount it somewhere else in the container which is not $HOME)
--from-home path1:path2:pathN which would tell toolbox enter to mount specific paths from host $HOME to the container home

So for example if I have source directories in ~/Projects, need to sign things with gpg and need ssh to connect to a server and my git config I could create a toolbox like this:

toolbox create --new-home --from-home Projects:.ssh:.gnupg:.gitconfig

So is this something would be accepted upstream? I would even work on this if the plan makes sense.

JaneSmith commented 3 years ago

Is there any update on this issue? It looks like onuruluag has implemented it separately back in April, as referenced above. Could those changes be pulled into Toolbox proper?

xrishox commented 2 years ago

i would like to see support added for this. not only do i not want my toolboxes polluting my home dir, sometimes i want different toolboxes to have configurations that would conflict with eachother. i also want my toolboxes to be entirely portable. i want to be able to export the image from 1 machine, import them into another machine and keep all of my configuration i've done. on top of that being able to fully back up these environments would be really nice too. that said i do like having access to my native home directory structure from inside the toolbox. i don't want my native home to not be mounted at all, i just don't want it to be the home inside of the toolbox. having functionality to support this would be absolutely awesome.

debarshiray commented 2 years ago

use toolbox for pet containers to deal with various programming projects, and I don't want my home directory to be cluttered by the containers.

Maybe use /opt or something for your projects?

swick commented 2 years ago

The problem isn't the source code but all the build and runtime tools which have state somewhere in $HOME.

debarshiray commented 2 years ago

The problem isn't the source code but all the build and runtime tools which have state somewhere in $HOME.

Can't those be redirected by overriding some default configuration for those toolchains somewhere? Just curious.

swick commented 2 years ago

Some do, some don't, sometimes they are used by other tools which don't let you pass the necessary config. Either way it requires much better knowledge of the tools you're using, more complexity you have to deal with and that makes it a lot more error prone. I understand the reluctance to add this feature but I don't really see another way to get a somewhat decent UX.

HunterGraubard commented 2 years ago

I desperately want this feature. This issue has been here for two years now and there's obviously still a lot of support for it. Can I ask why this issue is marked as Closed instead of Open? It was originally marked as Closed on the 18th of May, 2020, as a "duplicate" of #183. However, as has since been pointed out, they're separate issues:

Issue #183 is asking for the container to be isolated with no access to the home directory. This means the user's home directory is not mounted at all, for security reasons.
This issue #348 is asking for the container to have an adjustable home directory, possibly as a subdirectory within the user's normal home directory. This means the user's home directory is still mounted. This is not about security, but rather file organisation and "directing" processes within the toolbox towards a certain folder.

They're two separate issues, so it seems to me that it was invalid to Close it originally as a duplicate. Given the amount of discussion since then, the repeated requests for the feature, people even forking to try to implement it... Could this issue be Opened?

Earlier today I read a blog post from Christian F.K. Schaller, about Fedora's vision for the Linux Desktop. It talks about all the main modern innovations, such as OSTree, Flatpak, Silverblue, PipeWire, Wayland — and also Toolbox.

The article claims that using Toolbox for containers improves over traditional development by reducing cross-contamination between projects, as you can have one container for each project. Here's a direct quote:

In fact pet containers even improves on how we used to do things as they allow you to very easily maintain different environments for different projects. So for instance if you have two projects, hosted in two separate pet containers, where the two project depends on two different versions of python, then containers make that simple as it ensures that there is no risk of one of your projects ‘contaminating’ the others with its dependencies

They literally say "ensures that there is no risk". But this is objectively false, because of this specific issue. Sure, you can have separate containers... but if they're sharing the same home directory, there is indeed still cross-contamination. Look at what another person posted earlier in this discussion thread, about how building something that uses Rust polluted the home directory with 800+ MB of stuff. Is this not completely contradictory to the above quote? Now both containers are contaminated with others' dependencies.

The article concludes by saying that their vision is for these things to become more mainstream, and they want developers to embrace the pet container model. Well, I would love to embrace the pet container model! Absolutely... But this issue is the main thing holding me back from doing so. In fact, specifically because of this one issue, I spin up entire virtual machines as my "pet containers" instead. This is because I do not want my containers to have cross-contamination, and I do not want my normal home directory to be polluted by all sorts of random development junk. Having to use whole VMs is massively more inconvenient than what Toolbox would offer, but I have to because this crucial feature is not being implemented.

Please seriously consider this feature. I'm not a developer for this project and am not familiar with how it is implemented, but to me it seems like something incredibly simple to implement and maintain. It doesn't seem like a highly complex feature that needs thousands of lines of code and constant maintenance to keep working. It's a simple thing that would have an enormous impact on how this software can be used.

yilkalargaw commented 2 years ago

I think it has grown into a contention point and I have given up on it being implemented.

Before I go on more detail I want say that was and still kind of am a fan of this project and I understand how hard it is to maintain an opensource project in which people request features without suggesting ways of implementing it or without having to deal with test cases and build failures. But I am going to point out my opinions on this topic for the last time. I also want it to be known that all criticisms arise from my love of the idea behind the project because it think it is exceptionally brilliant one.

The feature is necessary if a wider array of people are to embrace pet containers. People have different work flows and they are going to hit this problem at one time or another. They have to deal with things that can't be installed using dnf like programming language package managers like cargo, programming language version managers like rbenv, rvm , nvm and the like. They might also have to work with programs like direnv which use hooks into .bashrc and .bash_profile to work properly. And many of these things can be dealt with if you can selectively share the home directory.
It also cleaner to sometimes have legacy projects you are working on to not litter things in the home directory.
It is also better for security reasons. I am not saying this is a cure to solving security problems with pet containers but I think it is one of the reasons people seek isolation through containers.
I have tried and suggested several hacks to achieve a similar thing but always ran into the wall because podman or fuse for some reason don't like it when give them alternate home directories through environment variables and they keep failing because they don't behave like regular bash scripts and crash in this edge case.
I have tried to read the toolbox source code but it has grown way more complicated than it needed to be. The go rewrite has made it harder for people to contribute if they do not know podman at an api level. I don't understand the necessity for rewriting using golang because CPU bottleneck or IO bottleneck has never been an issue with this program. People at the max run it several times a minute.
And lastly the maintainers have always disregarded the necessity of the feature. It might not be the actual reason but I think they have taken it as some kind of critique against the idea of their whole project or something which it is not. The number of people who run into this issue keeps amazing me.

Because of all the above issues I was not able to keep using toolbox. But I still stuck with the wonderful idea behind it all the idea of pet containers using podman. People have written several alternatives from simple ones that just manipulate command line arguments to podman to more complicated api level implementations. I am going to list out a few incase people running to this issue might find alternatives.

tlbx: a simpler bash alternative that uses default oci containers to run any distro but you can only use fedora, centos or redhat as the host. I have a fork at my repo which contains minor changes to solve some issues.
distrobox: more complete implementation and works on any distro, implemented using posix shell.
podenv: built around the idea of container apps but can do what you need.
nsbox: heard a lot of good things about it but have not used it but it looks really good. Has a whole documentation site
etui: I have not worked with it much but has good ideas behind it.

ps. there might be others I don't know about so we might share them to compare implementations and incase they might be helpful to people facing similar issues.

People reporting in issue #183, #331, #320 might find this useful

markstos commented 2 years ago

@yilkalargaw There's a typo in the tlbx link above. It should be: https://gitlab.com/uppercat/tlbx

yilkalargaw commented 2 years ago

Thanks @markstos I think I have corrected it.

maxboone commented 2 years ago

@yilkalargaw thank you for the write-up, couldn't agree more.

I do work for multiple teams inside our business, and every team / project that I need to run and test has it's own little quirks or patches. I used to run Windows Subsystem for Linux, and it was nice that you could just run completely separate (actually Hyper-V containerized) environments side-by-side.

I installed Fedora in the hopes that I could use toolbox in a similar way; Unfortunately the home-directory is shared by default indeed. I guess this can probably be overcome with some scripting here and there and/or using a non-default image. I will instead look into the alternatives you list.

The alternatives weren't stable enough and/or required similar hacks that can make this work on toolbox making me doubt about them.

maxboone commented 2 years ago

I think the hard part in this issue is that OCI-containers are created immutably. One of the main components of toolbox is that it has both a host-running management application and a special entrypoint that starts up the container configured properly and up-to-date according to the host set-up.

However, the home-directory is not mounted using this entrypoint. Instead, when toolbox create is run, the host-application checks which home-directories it should pass to the container and creates the mounts for these directories in the OCI-container (immutable) definition.

As far as I can tell there are two ways to go here (user = bark, container = yard):

Do not create direct mounts for the home directory when creating the container; Instead, mount the necessary home-directories of the users in /run/host/home/bark, have initContainer.go create directories on the image & symlink them on boot of the container (similar to existing approach for other exposed files).
Upon creation of a container, allow a flag (e.g. --private) that creates the yard::/home/bark mount but mounts it on the host to a configured directory (default host::/home/bark/.toolbox/home/yard/bark/...)

Considering security - as the toolbox runs under the same uid & gid as the host user, I wouldn't see leaving out a mount as secure. I also wouldn't go as far as allowing fully "private" containers, you can just as well use podman / docker / nerdctl without this. I also prefer option 1, as this keeps the door open to later mount the home directory (or even another directory) anyways.

For now, I won't have time to look into contributing to this project before May; I'm using the following lines in my .bashrc to have the containers use a subdirectory as their home:

# Check if we're running inside a toolbox
TOOLBOX_NAME=""
if [ -f "/run/.containerenv" ]; then
    TOOLBOX_NAME=$(sed -nr 's/^name="(.*)"$/\1/p' /run/.containerenv)
        HOME="$HOME/.toolbox/homedir/$TOOLBOX_NAME/$USER"
        mkdir -p "$HOME"
        bash
        exit 0
fi

rugk commented 2 years ago

@maxboone What is this /run/.containerenv file? It does not exist in my Fedora Silverblue 35 installation? (also not in a container)

maxboone commented 2 years ago

@maxboone What is this /run/.containerenv file? It does not exist in my Fedora Silverblue 35 installation? (also not in a container)

Hmm, I thought it was created by the toolbox init script but that seemingly only creates .toolboxenv. Let me check.

Seemingly that's a file that podman should create:

https://docs.podman.io/en/latest/markdown/podman-run.1.html

Unfortunately, the .toolboxenv file doesn't seem to contain any information on the environment itself; Probably the environment itself contains some info.

In any case, I have removed toolbox from my system and might fork the project to make it depend less on podman, work with docker & nerdctl as well and allow more customization. Using the HOME= wasn't respected as much as I expected to, some applications that i ran still just wrote to /home/$USER instead; Thus, for me it's really necessary to get rid of that mount.

tazihad commented 2 years ago

Why this issue is closed? I installed bunch of program inside toolbox. nvm, pipx, dotnet etc. Just to realize that all of this is corrupting my host home directory. And later found out that I have pip available in my host terminal. Even though pip doesn't come installed in Fedora Kinoite. This is weird. This issue makes the whole toolbox project baseless. Every toolbox container should come with separate homepage.

pyrotek45 commented 2 years ago

how is this not a standard feature. anyone who programs rust with a toolbox is going to have a polluted home folder like i found out mine was after installing rustup in a toolbox. big mistake. this needs to be added asap imho.

JaneSmith commented 2 years ago

@debarshiray You couldn't at least respond to all the extensive feedback and explain why this isn't planned? :(

mikezila commented 2 years ago

Could we at least get some rationale or an alternative solution to this problem rather than just a cowardly silent close? This is a very real shortfall that undermines the entire stated purpose of toolboxes. That there are so many proposed fixes and real examples as to why they're needed and yet we get no response is tone deaf.

tazihad commented 2 years ago

while having same home folder is convenient for average desktop user. But it's not ideal for developers. Who likes to try bunch of program to try in separate toolbox. So, I don't use toolbox in Fedora Kinoite anymore. I always use distrobox. You can have separate home in your choosing location.

xrishox commented 2 years ago

i'm also using silverblue and have also switched to distrobox. on top of supporting this feature it also natively and actively goes out of it's way to support running non-fedora distros inside of their containers.

inikishev commented 2 years ago

I think the hard part in this issue is that OCI-containers are created immutably. One of the main components of toolbox is that it has both a host-running management application and a special entrypoint that starts up the container configured properly and up-to-date according to the host set-up.

However, the home-directory is not mounted using this entrypoint. Instead, when toolbox create is run, the host-application checks which home-directories it should pass to the container and creates the mounts for these directories in the OCI-container (immutable) definition.

As far as I can tell there are two ways to go here (user = bark, container = yard):
1. Do not create direct mounts for the home directory when creating the container; Instead, mount the necessary home-directories of the users in `/run/host/home/bark`, have `initContainer.go` create directories on the image & symlink them on boot of the container (similar to existing approach for other exposed files).

2. Upon creation of a container, allow a flag (e.g. `--private`) that creates the `yard::/home/bark` mount but mounts it on the host to a configured directory (default `host::/home/bark/.toolbox/home/yard/bark/...`)
Considering security - as the toolbox runs under the same uid & gid as the host user, I wouldn't see leaving out a mount as secure. I also wouldn't go as far as allowing fully "private" containers, you can just as well use podman / docker / nerdctl without this. I also prefer option 1, as this keeps the door open to later mount the home directory (or even another directory) anyways.

For now, I won't have time to look into contributing to this project before May; I'm using the following lines in my .bashrc to have the containers use a subdirectory as their home:
# Check if we're running inside a toolbox
TOOLBOX_NAME=""
if [ -f "/run/.containerenv" ]; then
  TOOLBOX_NAME=$(sed -nr 's/^name="(.*)"$/\1/p' /run/.containerenv)
        HOME="$HOME/.toolbox/homedir/$TOOLBOX_NAME/$USER"
        mkdir -p "$HOME"
        bash
        exit 0
fi

is there any reason not to use this solution though? Also what is the correct way of using it, do I add it to the end of .bashrc?

inikishev commented 2 years ago

I think the hard part in this issue is that OCI-containers are created immutably. One of the main components of toolbox is that it has both a host-running management application and a special entrypoint that starts up the container configured properly and up-to-date according to the host set-up. However, the home-directory is not mounted using this entrypoint. Instead, when toolbox create is run, the host-application checks which home-directories it should pass to the container and creates the mounts for these directories in the OCI-container (immutable) definition. As far as I can tell there are two ways to go here (user = bark, container = yard):
1. Do not create direct mounts for the home directory when creating the container; Instead, mount the necessary home-directories of the users in `/run/host/home/bark`, have `initContainer.go` create directories on the image & symlink them on boot of the container (similar to existing approach for other exposed files).

2. Upon creation of a container, allow a flag (e.g. `--private`) that creates the `yard::/home/bark` mount but mounts it on the host to a configured directory (default `host::/home/bark/.toolbox/home/yard/bark/...`)
Considering security - as the toolbox runs under the same uid & gid as the host user, I wouldn't see leaving out a mount as secure. I also wouldn't go as far as allowing fully "private" containers, you can just as well use podman / docker / nerdctl without this. I also prefer option 1, as this keeps the door open to later mount the home directory (or even another directory) anyways. For now, I won't have time to look into contributing to this project before May; I'm using the following lines in my .bashrc to have the containers use a subdirectory as their home:
# Check if we're running inside a toolbox
TOOLBOX_NAME=""
if [ -f "/run/.containerenv" ]; then
    TOOLBOX_NAME=$(sed -nr 's/^name="(.*)"$/\1/p' /run/.containerenv)
        HOME="$HOME/.toolbox/homedir/$TOOLBOX_NAME/$USER"
        mkdir -p "$HOME"
        bash
        exit 0
fi
is there any reason not to use this solution though? Also what is the correct way of using it, do I add it to the end of .bashrc?

okay so although i dont really know what i am doing, from what i can tell it seems to work when you add it just before "unset rc". After doing that when I do cd ~ in a toolbox it goes to the folder. One issue is that after you enter a toolbox you need to write cd ~ to actually go to it's new home dir, I wonder if that is possible to do automatically. I tried adding "cd ~" to the script but I am assuming thats not how it works because it stopped the whole thing from working.

Anyway still is there any disadvantage to using that approach?

Enteleform commented 2 months ago

@debarshiray

I am going to close this issue so that the list of open issues doesn't balloon out of control, and continues to roughly reflect the items that are on the current to-do list. However, it will still be around for future reference.

Not having something on the immediate to-do list isn't the same thing as don't solve it. There's such a thing as priority.

It would be helpful to pick a primary thread (#183, #320, #331, #348, #1044, #1470, #1510) for this issue and reopen it, so that people can be notified when the issue is resolved. #183 and #348 are good candidates, as they have the most activity and dialog surrounding the issue.

It is apparent that this issue is affecting many people, as evidenced by the numerous duplicates of the issue and the extensive ongoing dialog.

Keeping an issue open would:

signal that the issue is intended to be resolved at some point
provide a singular notification channel for those who are currently resorting to workarounds or alternative solutions
mitigate further duplicates from being created

debarshiray commented 1 month ago

The primary issue is https://github.com/containers/toolbox/issues/183 Aren't you getting notifications from updates to it?

containers / toolbox

Adjust home path #348