Open stevengj opened 7 years ago
Similarly, there should be a way for users to specify which option they want to use when multiple are available, presumably via a TOML file?
The situtation perhaps is more complex. Though we might be able to ignore the complex case and get the low hanging fruit.
Etc etc, the constraints are fairly complex.
@oxinabox, that doesn't seem too much more complex to me. It just means:
A package can have multiple sets of optional dependencies.
Optional dependencies should include "none" as an option (in which case the package should disable the corresponding functionality).
Would this be mutually exclusive, with only one package allowed to actively satisfy the dependency at a time?
@tkelman, I don't think so. At Pkg.build
time, the package should be able to find out which (if any) of the optional dependencies were installed, and can then decide to use all of them or just one of them depending on how the package works.
But there should be a way for the user to indicate a preference if the package is just going to use one of the options.
I'm thinking of an interface something like the following. Suppose MyPackage wants at least one of three different FFT packages, and optionally can use a plotting package or can disable plotting if no such package is present. All of these optional packages are declared as optional=true
packages in the Pkg3 TOML file. In addition, you add section declaring the alternatives preference:
[option.fft]
packages = ["FFTW", "MKL", "FFTPACK"]
[option.plots]
packages = ["", "PyPlot", "Gadfly"] # "" means it is okay to install nothing
Then, in Pkg.add("MyPackage")
, if one of the options is not already present, it will install the default ("FFTW"
for fft
and nothing for plots
).
Pkg.option("MyPackage", :plots)
will return a list of all installed plots
options (or an empty list if none), in the default order ["PyPlot", "Gadfly"]
unless the user has changed the default. The build script for MyPackage will typically call this function to decide which option to configure with (if any).
Pkg.option("MyPackage", fft="MKL")
will change the fft
option to "MKL"
, installing "MKL"
if needed, and rebuilding MyPackage
if the option has changed; Pkg.option("MyPackage", :fft)
will now list "MKL"
first (returning ["MKL", "FFTW"]
if FFTW is also installed).
You can specify more than one package, e.g. Pkg.option("MyPackage", fft=("MKL","FFTPACK"))
, and it will install both of these dependencies. Pkg.option("MyPackage", :fft)
will return the dependencies with the user-specified ones first: ["MKL", "FFTPACK", "FFTW"]
You can also specify the option as keywords in Pkg.add
or Pkg.build
. e.g. Pkg.add("MyPackage", fft="MKL")
.
@KristofferC and I have been brainstorming and debating this for a while and so far the best scheme we've come up with is something like this made up example:
[deps]
ABC = "<uuid>"
Gtk = "<uuid>"
TclTk = "<uuid>"
Qt = "<uuid>"
XYZ = "<uuid>"
[alts]
graphics = ["Gtk", "TclTk", "Qt"]
[build.deps]
Builder = "<uuid>"
Clang = "<uuid>"
GCC = "<uuid>"
MSVC = "<uuid>"
[build.alts.compiler]
linux = ["GCC", "Clang"]
macos = ["Clang"]
windows = ["MSVC"]
[test.deps]
Test = "<uuid>"
QuickCheck = "<uuid>"
There are two different things at play here – alternatives and targets:
build
and test
are targets which add dependencies on top of the main set of dependencies.graphics
is a set of alternatives for the default target, letting you choose between one of three graphics backends – Gtk, TclTk and Qtcompiler
is a set of alternatives for the build
target that lets you choose a compiler in a platform-specific way – between GCC and Clang on Linux, Clang on macOS and MSVC on Windows.The general rules are that:
deps
is requires by the default target unless it appears on the RHS of a set an entry in the atls
section.alts
section is required in place of its RHS alternatives.ABC
, XYZ
and graphics
which may be one of Gtk
, TclTk
or Qt
Each target is a microcosm of the top-level with its own deps
and alts
sections. These can be merged as dicts, as in:
target_deps = merge(project["deps"], project[target]["deps"])
target_alts = merge(project["alts"], project[target]["alts"])
If there are any key collisions it should be an error – targets allowed to override the identity of a package. A generalization of this maybe this:
target_project = recursive_merge(project, project[target])
where recursive_merge
recurses into dictionary values and errors on key collisions. The resulting target_project
can be saved as a project file and used for executing the target – build
, test
, etc.
The motivation for naming sets of alternatives is that it allows configuration files to specify them as in:
[config.SomePackage]
graphics = "Qt"
compiler = "Clang"
Maybe its worth spelling alts
out as I think it wouldn't be used that much? Unlike deps
it's not a "word". Another word would be choices
or choiceof
, which reads well in above example.
I was originally writing out alternatives
but I got tired of typing it over again.
I find the compiler
to be a bit out of place … how is that fundamentally different from any other non-Julia dependency? Hopefully, the vast majority of projects will use something like BinaryProvider.jl and won't require a compiler.
It's just an example of an system-specific alternative. Replace it with
[build.alts.glam]
linux = ["Glow", "Sparkle"]
macos = ["Sparkle"]
windows = ["Glint"]
if the specifics are bothering you.
How does the user choose the alternative to be used?
It would be nice to support keywords in Pkg3.add
, e.g. Pkg3.add("Foo", graphics="Tk", glam="Sparkle")
, in addition to any kind of terminal-specific interactive menu.
There should also be a way for the config file to specify a default choice, so that if you have a bunch of complicated options it will automatically select a reasonable one and only sophisticated users can opt to select a different option.
You also want to be able to change the option later, e.g. by calling Pkg.build
again, and the options should also be exposed to the package itself so that it can query them at build
time. See also my proposals in #38.
I would expect that earlier alternatives are preferred so the ordering gives the preference. I don't see any reason why we can't support keyword arguments for picking alternatives. We can also interactively prompt people for which one they want.
Should the config file be able to specify whether the first one is picked silently if the user doesn't specify (i.e. if there is a reasonable default), or if the user is always prompted for a choice?
I also find it confusing how you are specifying platform-dependent options. What determines which options have platform-specific choices (build.alts.glam
) and which ones don't (graphics
)? Shouldn't all options potentially be either platform-specific or platform-agnostic?
And why is it [alts]
and [build.alts.glam]
? Shouldn't it be [alts.graphics]
for consistency?
If the dict value for an alternative is an array then it's platform-agnostic, if it's a dict of arrays, then it's platform-specific and the keys are used to choose which set of options apply.
I wonder whether we want Pkg3 to contain something like Debian alternatives: a package A can depend on package B or package C or package D.
For example, the Plots.jl package might want to depend on having at least one plotting package installed (cc @tbreloff).
Or, now that FFTW.jl is being split off into its own package, and that package in turn will probably depend on an AbstractFFTs package (JuliaMath/FFTW.jl#2), other packages may want to depend on having some package installed that implements FFTs following the AbstractFFT interface (e.g. FFTW, or a native Julia FFT, or FFTPACK, or MKL, etcetera).