Package options

[stevengj]

It would be nice to have uniform way to specify package configuration options. Currently, I'm using environment variables for this (e.g. PYTHON in PyCall), but that is somewhat non-Julian. More importantly, it requires some manual effort for me to save the option in a file so that it is "remembered" when you do Pkg.update etcetera even if the environment variable is no longer set. Many packages will get this wrong, and even if they get it right it is a lot of duplication of effort.

My suggestion would be that the package TOML file should have something like:

[option.python]
value = "python3" # default

The user interface would be something like:

• Pkg.option("PyCall", :python) — return the current choice (saved in a file somewhere)
• Pkg.option("PyCall", python="python2") — change the option, maybe rebuilding automatically unless this is called by the package during the build
• Pkg.add and Pkg.build will accept python="python2" keyword arguments to set options when installing/building.

The value of all options would be saved in an options.toml file somewhere.

A related mechanism would be used for package alternatives, see #37.

[stevengj]

The lack of a good way to do this is starting to be a serious problem.

Right now, e.g. PyCall and Conda save preferences (like which python version to use) in the deps directory — in 0.6, this allowed it to "remember" your preferences the next time it was updated. In the new package design, this no longer works because packages are not updated in-place.

I could install the preferences into ~/.julia/config, but then you run into global namespace issues and an inability to have multiple configurations living side by side.

@StefanKarpinski and @KristofferC, do you have any ideas about where persistent preferences should be stored?

[ChrisRackauckas]

@StefanKarpinski has an idea written out here: https://github.com/JuliaLang/Pkg.jl/issues/458#issuecomment-403104562

[stevengj]

That explicitly deals with only project-specific configuration. I disagree with Stefan that package-specific configuration is unneeded. If the user installs PyCall, how do they specify the Python version in a way that will be remembered across updates? Asking them to create a project just to do this seems like a nonstarter.

It seems like there needs to be way to store package options persistently in a given environment, perhaps in a way that is inherited by projects created within that environment.

[StefanKarpinski]

I never said that package-specific configuration was not needed. I have said that package configuration must be a global property controlled by an application or global environment, not by other packages since packages are reusable.

[simonbyrne]

well, you could allow it in packages, but let the projects have primacy.

[StefanKarpinski]

That means that packages will get tested one way but run a different way. Seems like a bad idea.

[simonbyrne]

I was just envisioning the package-provided one to be the "default" (though of course, that logic could be handled in the Pkg.build as well).

We should provide a way to override these at the command line or via environment variables, so that it is easy to set up a test matrix.

[oxinabox]

So I am clear, would it be envisioned that this be used for something like my configuration needs for DataDeps? http://white.ucc.asn.au/DataDeps.jl/latest/z10-for-end-users.html#Configuration-1

That is 5 environment variables, and it is about to get a 6th to control how often download progress is displayed.

These things are configurable at run time.

---

I know C# and Java include built in configuration libraries. The C# one is extensive. Where each project does ship with a app.config file or similar, that can be trivially read using something like Config["KEY"]. (And in C# you can do a ton of stuff to automatically map it to object properties etc.)

[tkf]

Given that https://github.com/JuliaLang/julia/pull/37595 is merged and we (will) have https://github.com/JuliaPackaging/Preferences.jl, I think we can close it now?