standard location for user binaries like ~/bin/, so installers don't write to oshrc/oilrc

[andychu]

pip seems to have a problem with this

https://www.gdbgui.com/installation/

https://pypi.org/project/userpath/

I want to install some "polyfills" too, and that is an issue.

---

maybe the default install script should write an oshrc that prepends ~/bin to $PATH ?

[andychu]

Tools that edit the .bashrc

• pipx
• nix for nix-profile
• I think OPAM for putting OCaml compilers in the $PATH
• rustup, the standard way to install Rust: https://www.rust-lang.org/tools/install

I think we need a protocol for this... symlink to ~/bin ?

Related to #588

[andychu]

This came up in an e-mail about config file evaluation and shvar PATH='' { ... }

I think we might need a META_PATH or something, for #848

I guess there are two separate goals

• It should be possible to write an installer for a user using Oil that is not super invasive. It shouldn't have to write to .bashrc or oilrc. In fact we should prevent that if we can!
  • But this doesn't do anything for users of other shells on the system
• More ambitious: provide a packaging and distribution format. A shell package manager

And another requirement here is to install binaries without root privileges. That seems to be standard these days, whereas it didn't used to be

[bar-g]

Interesting.

to install binaries without root privileges. That seems to be standard these days, whereas it didn't used to be

It's a large security problem, though, to allow arbitrary placement and execution of executable binaries.

In unix/linux the risk is traditionally dealt with by only allowing the execution of files that have been selectively approved (granted executable file permissions) and moved to an immutable location (e.g. /usr/local/bin for non-distro-packages). The approvement for system wide execution requires root authentication.

This approach could nicely lend itself for user installed executables very similarly, here only checking user authentication, though (i.e sudo/doas? takes the user's password and the approved code is moved into the otherwise not writable ~/bin and made executable.)

Oilshell already brings, or can bring improvements and simplifications, to protect some very important corners in a shell:

• Arbitrary code execution by external data handled within scripts.
  • static variable expansion, eval,...
• Arbitrary code execution by external files (downloads, redirect exploits, ...) present within a user account.
  • support for some "protected ~/bin" scheme (not directly-writable ~/bin directory, listed in not directly-mutable $PATH)
• Arbitrary overriding of higher privileged (root approved) code by lesser privileged (user approved) or even not approved code. Privilege escalation!
  • su/sudo/doas as keywords (simple proxies?) and blocked as buildin,functions,...?
  • oilrc only read from protected locations (~/bin/oilrc, and it sourcing only within ~/bin and /etc?)
  • $PATH only settable in oilrc, and only directly mutable through protected sudo/doas, otherwise const.

Some complementary, and not shell-bound, but still related solutions are:

• "noexec" mounts
• "firejail oil"

[bar-g]

Concerning that more ambitious package manager aspect: Zeroinstall seems to solve the problem cross-platform, already. With its hashed names it even allows to have the files automatically installed into a protected directory that is shared system-wide. https://docs.0install.net/details/sharing/

Once there is an established scheme for a protected ~/bin (single-user-installs), zeroinstall could also support to move single-user cached implementations into the protected ~/bin directory.

Making one's project's sources readily available and compilable with the zeroinstall tools (incl. crypto-secured downloads for the supply chain) is even quite easy. See https://docs.0install.net/tools/0compile/developers/

[bar-g]

Here are also good explanations of how different installation schemes can work with different guarantees based on the same resource permission system: https://docs.0install.net/details/security/

Could be a good read to clean up above two remaining shady security corners of the shell (protected ~/bin, and protected privilege escalation tools).

[andychu]

Do you have any good experiences with 0install?

I heard of 0install like 10-15 years ago ... but I never heard anyone use it?

I think the issue is bootstrapping (how do I install the installer)

and the fact that Ubuntu and even Debian want to control their software distribution. It hard for a non-OS to control the installation situation over 10-15 years ...

e.g. Red Hat and Ubuntu both have their own container-based packaging now too ..

---

I think the general problem with this issue, and which your comment hints at, is that it quickly expands into Write a Package Manager

I do think that is complementary to Oil, but I tried to do it once, and it's a HUGE HUGE job ...

---

But still maybe we can do something very small without trying to boil the ocean

Another problem is that people use multiple shells on the same machine. So Oil could have ~/bin in PATH, but bash might not, and that could break bash scripts.

[andychu]

Also this ties into #1075

In some sense that is the opposite. If you develop INSIDE A CONTAINER, then the issues are mitigated. Package managesr can install in to /bin but it's NOT the global /bin

So then we don't have to "boil the ocean".

---

The key is to bring the oilrc inside the container nicely! So you don't lose all your settings.

Ditto for vimrc and such.

And also you don't want to lose GUIs / X / ssh-agent

It's a tough problem

[bar-g]

for a non-OS

what do you mean here?

[andychu]

Just saying that zeroinstall is not an OS, while Ubuntu and Windows are.

So yeah basically I think I haven't seen it used in ~15 years because it has the "who installs the installer" problem.

If you're an OS, then you don't have that problem. It comes with the OS itself

So the native OS mechanisms will always be more fundamental, because you use them to install a package manager / installer

(If Oil is packaged with an OS, then it will be a native OS mechanism... but that's a long way off. Even bash was not that popular until ~10-15 years into its life)

[bar-g]

Ok, sure releasing distro-independent packages only makes sense for projects until distro-packages are available, and, if that package manager implements package signature verification that's better than manual downloads and verifications.

0install is included in several distros (e.g. https://get.0install.net/#linux-ubuntu) and can check signatures. So, installing the installer may be just like any other package. Unfortunately, following the packaging docs didn't produce an easy success: https://github.com/0install/0publish/issues/6

Nix-bin seems in debian but that does not seem to be indicated on the nix install page. And nix does not seem to support signed source packages: https://github.com/NixOS/nix/issues/613 https://docs.0install.net/about/comparison/#nix

Another thing that the language specific package installers seems to do is managing different environments. In this direction I noticed this shell-based approach: https://github.com/ilyash/E

Interestingly, your prospective Oil features to help tame those solutions (rc file access) seem to correlate to the ones to help tame against further breakage: ~/bin can be included in $PATH if safe, as in https://github.com/oilshell/oil/issues/1143 (and a firejail configuration could be used to provide safety).

[sullyj3]

From the xdg spec:

User-specific executable files may be stored in $HOME/.local/bin. Distributions should ensure this directory shows up in the UNIX $PATH environment variable, at an appropriate place.

https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html