Thoughts about management of the environment

[ghost]

Dumping here the result of various discussions we had with about environment variables, for future reference.

Currently, Dune passes through the environment it is executed in to commands it runs. The only management of the environment we do is allowing users to declare dependency on environment variables, so that actions are re-executed when such environment variables change.

There are several issues we know about regarding the environment:

• we don't have a good way of detecting when an action depends on an environment variable without declaring it. When this happens, we will sometimes not re-execute an action when we should instead do it
• Dune sets various environment variable to local paths, for instance it sets PATH so that commands can access binaries with a public name or binaries locally added to the PATH via (env (_ (binaries ...))). It does something similar for libraries by setting OCAMLPATH. When an aciton is sandboxed, such variables still point to paths outside of the sandbox, allowing actions to "escape" the sandbox and see things they are not supposed to see

For the first issue, we discussed and/or tested a few ideas:

• making Dune very strict, by cleaning the environment and only passing variables declared as dependencies. This might be tedious to actionate, as for instance the libc or some other library might be reading some variable to be set that we don't know about
• taking into account the whole environment when memoizing command. That would mean that changing a single variable would cause Dune to re-execute all actions, which is probably too extreme
• doing something in the middle: being strict about variables Dune knows about. For instance, if the user sets X via the env stanza, then any action declaring a dependency on X will see it, and X will be clearer from the environment of actions that don't declare a dependency on it

As always, any behaviour change should be tied to the version of the Dune language to ensure good backward compatibility.

We are currently passing around the environment as a string map, and overall it feels like we should probably use some more structured representation.

[bobot]

(For the second issue we discussed splitting those directories, for example one directory for each binary or for each package so that we can create a PATH variable which really represent the binaries declared as dependencies (similarly to nix))