Closed noverby closed 8 months ago
This I am a bit torn on.
On one hand, if a flake has a lib output, it is reasonable to put it in the overlay to allow easier use, and avoid the flake having to declare that.
On the other hand, there a few issues.
For one, the lib would need to be composable with nixpkgs lib. Most flakes I have seen with lib outputs, while this would work, are not designed with that in mind. Usual use would be use lib
for nixpkgs and flake-name.lib
for that flake's lib separately. Even if a flake does not use a name currently in nixpkgs, a future update to nixpkgs could cause the overlay to mask an incompatible function and cause hard to debug issues. Flakes that do intend to compose also usually don't put their functions in the top-level of lib but in lib.flake-name
. This would be the better general approach but requires knowing that that flake-name name should be. As an edge case, a flake lib which is made extensible and relies on extending itself could malfunction when combined with nixpkgs lib, as we'd have to use nixpkgs' extensibility.
Second, with packages, there is a distinct difference between using the overlay and using the flake package outputs. Flake package outputs are built with the flake's deps, but overlays use deps as supplied by the user. For lib code, the functions would be the exact same whether used from the flake output or the overlay, so theres much less value. To make them useful, the lib would have to be defined as an overlay that uses lib functions from prev
and have the regular overlay generated from that. This would be non-standard and wouldn't make sense as default imo.
Third, as a user of the flake, I'd find it surprising if the overlay modified lib, especially because it could conflict with my own lib modifications.
Currently to add lib to the overlay, namespacing under flake-name, you could do something like:
{
inputs.flakelight.url = "github:accelbread/flakelight";
outputs = { flakelight, ... }:
flakelight ./. rec {
pkgs.hello = { hello }: hello;
lib = { test = x: x; };
overlay = final: prev: { lib = prev.lib.extend (final: prev: { flake-name = lib; });};
};
}
The above unfortunately is not simple, but would have both lib.flake-name
and the package in the output overlay. We could make a libOverlay option to directly set lib overlays though that would be non-standard. Also I'm not sure putting lib in the overlay is a good idea in the first place.
So I am leaning towards not doing lib automatically since its unclear what the automatic thing to do is. There does not seem to be a clear "good way" to do it. I could be convinced otherwise though.
Maybe there could be an option to expose lib attributes, like they do in crane with e.g. mkCargoDerivation
: https://github.com/ipetkov/crane/blob/master/lib/cargoBuild.nix
Yeah, theres already an option for exposing a lib attribute: https://github.com/accelbread/flakelight/blob/master/api_guide.md#lib-1. It also supports autoloading.
Though crane is also a bad example since it breaks convention by having crane.lib.${system}
lib outputs instead of the conventional single lib output.
To have a normal lib output, can just use the flakelight option. If you want to have per-system lib outputs like crane, can use the perSystem option to set lib.
@noverby But if you use the lib
option, will all the lib attributes then be available globally?
E.g. if you define mkCustomDerivation
in nix/lib
, can it then be used in nix/packages
:
# pkg1.nix
{mkCustomDerivation, ...}: mkCustomDerivation { ... }
You would be able to do the following:
# pkg1.nix
{outputs, ...}: outputs.lib.mkCustomDerivation { ... }
Could also use self, like:
# pkg1.nix
{inputs, ...}: inputs.self.lib.mkCustomDerivation { ... }
Though I would avoid using self, since when using self its easy to trigger inf recursion.
Though, if you want to access functions directly through the package set, you can use withOverlays
See flakelight-rust, which uses withOverlays to add crane stuff to the package set: https://github.com/accelbread/flakelight-rust/blob/master/flakelight-rust.nix
To use withOverlays to have all your lib stuff available, would need something like:
{
inputs.flakelight.url = "github:accelbread/flakelight";
outputs = { flakelight, ... }:
flakelight ./. {
withOverlays = final: prev: prev.outputs.lib;
};
}
With that, your last example would work with getting mkCustomDerivation directly from the pkgs set.
@accelbread Yeah you are right about infinite recursion :) By adding lib to overlays, as you suggested, it causes the following infinite recursion error:
It seems to work fine, when just defining lib/default.nix
, but if I define any other file e.g. lib/helper.nix
, then it will result in an infinite recursion no matter the content of lib/helper.nix
.
Ah yeah; outputs
is from the module system which needs withOverlays
to compute which attrs are output, so thats a loop.
Using outputs.lib.mkCustomDerivation
without withOverlays should be fine though.
Is nix/lib
suppose to support functions like this?
# nix/lib/default.nix
{src, ...}
: rec {
rustToolchain = {pkgs}:
pkgs.pkgsBuildHost.rust-bin.fromRustupToolchainFile (src + /rust-toolchain.toml);
rustToolchainForAarch64 = {pkgs}:
pkgs.aarch64.pkgsBuildHost.rust-bin.fromRustupToolchainFile (src + /rust-toolchain.toml);
craneLib = {
pkgs,
crane,
}:
(crane.mkLib pkgs).overrideToolchain rustToolchain;
craneLibAarch64 = {
pkgs,
crane,
}:
(crane.mkLib pkgs.aarch64).overrideToolchain rustToolchainForAarch64;
}
It can support any structure for lib; it only requires that lib is an attrset (or a function when autoloading). If using nix/lib/default.nix
, whatever is in that file will be exported as the lib attribute if it is an attrset, and it will be passed module args if it is a function.
So yes, that should work.
If crane is a flake input, can also use { src, inputs, ... }
for the params and use inputs.crane
.
Though if its only intended for use inside the flake (not exporting to other flakes), instead of using lib, I'd try:
# nix/withOverlays.nix
final: prev:
rec {
rustToolchain =
final.pkgsBuildHost.rust-bin.fromRustupToolchainFile (final.src + /rust-toolchain.toml);
rustToolchainForAarch64 =
final.aarch64.pkgsBuildHost.rust-bin.fromRustupToolchainFile (final.src + /rust-toolchain.toml);
craneLib =
(final.inputs.crane.mkLib final).overrideToolchain rustToolchain;
craneLibAarch64 = (final.crane.mkLib final.aarch64).overrideToolchain rustToolchainForAarch64;
}
That way you get direct access in all package defs and pkgs args.
@accelbread If I move the overlay to a withOverlays
subdir like nix/withOverlays/rust.nix
, then I get the error:
A definition for option `withOverlays' is not of type `(list of nixpkgs overlay) or nixpkgs overlay convertible to it'. Definition values:
withOverlays
expects an overlay or a list of overlays. nix/withOverlays/rust.nix
tries to set it to { rust = <contents of nix/withOverlays/rust.nix>;}
which won't work. If you want to split your overlay into multiple overlays, each in their own file (for example nix/withOverlays/a.nix
and nix/withOverlays/b.nix
), then you can add the following file:
# nix/withOverlays/default.nix
[
(import ./a.nix)
(import ./b.nix)
]
That way withOverlays is set to that list, and you can organize the multiple overlay files as you wish.
Or if you have just the one overlay and just want it to be in a folder, then just naming it nix/withOverlays/default.nix
also works.
Thanks @accelbread!
Using withOverlays seems like a proper solution, so I will close this issue.
Right now, packages are available in
overlay.default
.Wouldn't it make sense to also expose
lib
inoverlay.default
, so you can define values and helper functions, that can be used globally?