epruesse
commented
6 years ago In general, I think, this whole thing is mainly applicable to packages producing binary object executables and libraries. Dynamically compiled or interpreted software doesn't fit the model very well (until we can ship the .pyc without the .py).
Typical splits
The separation is usually done primarily by "use case". Do I want to install package as User (i.e. I want to run things provided), as Package (I need to link to things provided), as Maintainer (I want to build packages), as Tester (I want to debug package), or as Developer (I want to write dependent code).
Application package
- XXX-dev - for Maintainers
$PREFIX/include/$PREFIX/lib/**.{a,la}
- XXX-doc - for Developers
$PREFIX/share/doc/,$PREFIX/share/man/
- XXX-dbg - for Testers
$PREFIX/lib/debug/
- XXX - for Users
$PREFIX/
To reduce storage needs for distribution, this one is also common:
- XXX-data - shared among architectures
$PREFIX/share/
Library package
For packages containing major libraries:
- libXXX - for Packages
$PREFIX/lib/**.{so,dylib}
- libXXX-NNN - for "older" Packages
$PREFIX/lib/**.so.NNN$PREFIX/lib/**.NNN.dylib
- libXXX-tools - for Users
$PREFIX/bin
These are pretty much what the main stream distros use commonly. Given their level of experience, I think it's a good start to just go with it. There are plenty of reasons. I'll make a case if you want. A good place to start might be the debmake manual as the build tool for much of the dpkg based distros.
Optional GUI package
One additional case is common if part of a package's functionality requires a large set of dependencies to be installed, e.g. in case of optional GUI features:
- XXX-common - shared data
- XXX-gtk3 - GTK3 gui
- XXX-qt5 - QT5 gui
- XXX - CLI only
Automation
With regards to automation from conda-smithy - the more the better. I don't know how easy this is to implement, as much is in the hands of conda-build - also, exceptions would have to be accommodated somehow. Technically, if the package installs it's files according to FHS, the -dev, -doc and -dbg parts are easy enough to auto-extract, leaving the rest for the primary package. The -dbg part should be extracted fully automatically as it's too difficult to get right manually and too easy to automate. Some of the build/host/run dependencies could also be derived I believe. I'm not very familiar with the new host: concept yet, so it's hard to say how that plays into it all. Generally, XXX-dev needs libXXX and all other *-dev packages from the build requirements. XXX needs libXXX if present and the lib* packages from build.
mbargull
scopatz
ocefpaf
msarahan
hmaarrfk
xhochy
jakirkham
ericdill
mariusvniekerk
In
conda-build3 it is possible to have multiple package outputs.This can be useful if a build requires to compile multiple pieces of source together, but is better distributed as separate packages after the fact (e.g. LLVM, gcc, APR, etc.).
However another use case is distributing multiple components of a package with different contents (e.g. devel, debug, so, src, docs, etc.). This also came up in issue ( https://github.com/conda/conda-build/issues/2363 ). This would be useful for a lot of packages where users want extra features (e.g. debug), but also want to have a lean version for Docker images and other deployments (e.g. so). It's also possible to generate a legacy metapackage, which pulls all of the contents back together.
It would be good to figure out what sort of split packages we might want to generate in conda-forge. Also would be good to discuss how we want to integrate their use into other downstream conda-forge packages. Would also be worth thinking about whether there will be any effect on our tooling (e.g.
conda-smithy).cc @conda-forge/core @epruesse