Context

The overwhelming majority of packages in the Nix ecosystem did not come from the Nix ecosystem. As such, the projects these packages are made to provide tend to use incompatible organization philosophies like version numbering.

Even so, users expect Nix to provide multiple instances of packages that are delineated by version number.

Nix Design

In Nix, a package is described in two places:

The derivation: Somewhere in nixpkgs/pkgs is a default.nix that describes where to get the source, how to build it, and what files to keep.
The global attribute path: The package name is defined in pkgs/top-level/all-packages.nix. The name evaluates a nix expression using the derivation defined in nixpkgs/pkgs/somewhere/package-name/default.nix.

It is assumed that a derivation will be built from the latest source available. To update a package, the derivation is edited to build from a newer source version, tested, and the changes to the derivation are added to a git PR to be merged with nixpkgs/master.

It isn't uncommon for a derivation to depend on a different version of a package than what is provided by its default.nix. For example, many C/C++ programs do not build with the latest GCC.

When different versions of the same project are provided, they each need a unique derivation file and attribute path name.

It is recommended in the manual that extra package versions are created as separate derivations alongside the default derivation, and the version name is appended to the package name:

If there is only one version of a package, its Nix expression should be named e2fsprogs/default.nix. If there are multiple versions, this should be reflected in the filename, e.g. e2fsprogs/1.41.8.nix and e2fsprogs/1.41.9.nix. The version in the filename should leave out unnecessary detail.

All versions of a package must be included in all-packages.nix to make sure that they evaluate correctly.

For example, there currently exist the attribute paths gcc6, gcc7, gcc8, gcc9, and gcc10 alongside gcc, which provides gcc 9.2.0.

Nix Usage

When installing a package, a user will generally refer to its attribute path name, as defined in all-packages.nix.

When a user updates their system with nixos-rebuild --upgrade [switch|boot] or nix-env -u [packagename|*], nix gets whatever derivations are defined in the latest nixpkgs commit.

NOTE: This is made a little more complicated by channels. Just remember that channels refer to git branch names.

Problem

It is fairly common for a user to want to use a different package version than the default one provided in their nix channel. In order to use it, the user must find it.

There are three ways this can be accomplished:

If there is already a derivation provided by the user's channel, the user need only find its attribute path name. This has gotten easier with nix search, but isn't always reliable. Usually, packages follow the naming convention of appending the version to the package name, but this is not always the case.
If there is not a derivation provided in the user's channel, the user must find a hash that points to the desired derivation version, and use pinning to refer to the package by hash. The hash can be found using this third-party tool, and used as described here.
If there is not a derivation that provides the desired package version, the user much create their own. Suddenly, a user who just wanted to use a simple and elegant package manager needs to learn how to write a nix derivation, or at least how (and where) to modify an existing one and use that. This is usually done by checking out nixpkgs/master, and either altering the existing derivation (or a copy) to one that builds the desired package version. This method is described in the manual here.

None of these options is obvious, clear, canonical, and easy to use.

Discussion has been happening on this topic for about five years (off and on) here: #9682

it's pretty obvious to me nix should have a simpler way for users to install a version not in their nixpkgs handling all the needed logic in a transparent way, i.e. i just say pname-version, did not find it in active nixpkgs/input? automagically try finding older definitions someway and do the logic someway... found derivation definition and not in the cache? nix says "we don't support pname-version you're on your own" and builds it. and for those who want more control to manually keep using overlays/overrides, different input revisions for nixpkgs, etc.

another thing is that nix just collides binaries when there are different versions with same binary name. i think for that nix should instead automatically append version to the binary name (better yet, keep the collided binary but symlink the binary to pname-version) so that it will be available to PATH. well appending version only would not solve the issue, we would need the hash as well pname-version-hash, or part of that hash. unison-lang.org might shred some light into this issue as well.

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nixpkgss-current-development-workflow-is-not-sustainable/18741/62

The root reason for me is; the maintainers seem indifferent to disscussions around fixing foundational problems that totally undermine reliability/stability/purity.

The issue with a particular package ? => just create an issue on the package manager (nixpkg) repo and want to publish a tiny update to your package? => you need a PR against nixpkg directly

were just nails in the coffin below the flashing "this is un-maintenable/un-scaleable" sign 😕

I'm excited to announce these^ statements may no longer be true as of that conversation in the comment above Nixpkgs current development workflow is not sustainable.

Update for this Issue

Version search is a small piece of a larger issue: general package search-ability. And I'm hopeful the discussion/proposal here will get some traction since it paves the way for solving this problem and the maintainability problem.

I should mention that my old statement:

How can I possibly care about the new flakes system when there isn't even a canonical name/version for packages?

has aged quite poorly. Now that I have much more experience and understand flakes, I can say the Nix overhaul I was talking about is (or can be) flakes. And core contributors have been working on that since before this issue was opened. Flakes are a standardization of package format, and this proposal is trying get searchable fields added to the format.

This is something of a +1, but expressed as a (re?)statement of a use-case, which I haven't seen, in black and white, in this issue or its progenitor (but which I may very well have missed in the epic flow). This is related to @jeff-hykin 's point in a comment above.

Say I have a collection of nix-installed software that works for my project, now, and I want to be able to reproduce that in future or tell someone else how to. How do I do that? That ‘tell someone else’ could be me writing in a publication ‘download this published data from <URL>, then (install NixPkgs and) install tools A, B and C using nix-shell my-magic-software-set.nix, and you will see what I'm seeing and can reproduce my research’. I want those instructions to be good for at least ten years. How do I assemble the contents of my-magic-software-set.nix?

If I have a working suite of software versions just now (but I know the previous release had a bug and I'm worried the next one will have a regression), I can look at the $PATH to work out the hash of the derivation and then I can write that down somewhere, and later, if I'm understanding Lazamar's site correctly, use that to retrieve that particular version, and all of the dependencies of that specific version. I might end up rebuilding the world, but that's OK. That's all a bit DIY, though – I'd hope that NixPkgs would make this a lot easier, and that it was documented somewhere other than a github issue.

This process is what I personally thought was NixPkgs USP, and the fact that I couldn't find anything about it in the manual was puzzling me greatly until I tried searching the manual with google, and stumbled across the progenitor issue here. To be clear: I long believed that the above scenario was the point of NixPkgs.

Note: despite the issue title, this isn't a question about versions, as such. One may agree with commenters above, that ‘versions don't really mean anything’, or with the less extreme position that ‘at least they're useful as searchable metadata’. I don't necessarily care what versions are in my magic-software-set, I just have to freeze them, for the sake of my own sanity and that of my article's readers.

The above assumes I get a hash by carefully storing it when I've got a working version (that's what notes/research-logs/record-keeping is for). It would be nice to be able to search (including searching versions-as-metadata) for what was the current versions when ‘damn – I'm sure this was working six months ago – what version was Nix-current then?’ Lazamar's database can help here, but I find myself here, trying to work out how to unearth a package which isn't one that they cover. And as others have said, one is surprised to have to rely on a third-party site for this.

Despite the length of this issue (and indeed of this comment), I'm still convinced that I'm missing something obvious, and that someone is about to say that this particular use-case is already trivially satisfied by X, Y and Z (where none of X, Y or Z are ‘clone the entire git repo’). I'm sure that TFM is trying to tell me this, but it assumes I am more comfortable with the Nix language than I am (it's clearly a great language, and I'm sure it'll be fun to learn at some point, but that might be a project for another day).

Say I have a collection of nix-installed software that works for my project, now, and I want to be able to reproduce that in future or tell someone else how to. How do I do that? That ‘tell someone else’ could be me writing in a publication ‘download this published data from <URL>, then (install NixPkgs and) install tools A, B and C using nix-shell my-magic-software-set.nix, and you will see what I'm seeing and can reproduce my research’. I want those instructions to be good for at least ten years.

What about statically compiling the program and storing the executable? I imagine that could be cheaper in both effort and storage space than directly solving this ticket's posited problem and needing to keep all the (transitive!) dependencies around somewhere (at least all the source code, which counts as dependencies in Nix). I think (or am I wrong?) that currently the NixOS Foundation is bearing all the costs of storing built nixpkgs packages indefinitely, but I don't suppose that can continue for ever.

What about statically compiling the program and storing the executable?

Static compiles might indeed be a possibility in some cases, though in the case of macOS, which relies heavily on dynamic libraries, I suspect building a ‘static’ executable could be a project in itself (though this isn't something I've investigated in depth).

The problem is that that wouldn't deal with the case of publishing a recipe (and that's literally publishing, in a journal article): ‘do X and you will be able to reproduce these results yourself.’

Re storage: I wouldn't expect to be able to download packages of historical derivations, merely the link to the location of the source (third-party location), the patches and build recipe (in NixPkgs), and the transitive links to the dependencies (also NixPkgs). My understanding is that those patches and build recipe are what are hosted by NixPkgs, not the source. (But I'm commenting in some ignorance here, since I'm aware that Nix would be a lovely rabbit-hole to explore, so I'm being cautious about how far I stray behind the front desk.)

Obviously, this depends on a third-party source location not changing, and that can't be guaranteed. There's a different research-reproducibility rabbit-hole we could run down here, but if a build broke because the source had disappeared, then boo-hoo, but it's not NixPkgs fault.

I'm still convinced that I'm missing something obvious, and that someone is about to say that this particular use-case is already trivially satisfied by

@nxg

You're not missing anything if you're aware of what's below.

You can use nix-pinning to get old versions (all results on lazamars site show the code for pinning). But, there is a risk. If package1 is from nixpkg commit1 and package2 is from nixpkgs commit2, there is no guarantee they will work together. For standalone binarys that's fine, but for packages that interact (ex: package1=python3, package2=numpy) it can result is terrible dynamic/shared library issues. A lot of that isn't nixpkgs fault, it's just the bad modern system design we have, like the recent breakage that glibc caused. Additionally there is no guarantee that a package will behave the same on a different system (linux/mac).

If all the packages for a project are from the same nixpkg commit, then it is very realistic to freeze all of them. If an example would help, let me know and I'll provide the code. Create a shell.nix file, use the mkShell tool within the nix language, load a pinned nixpkgs (as a variable), then list all the packages in build inputs. Once that file is made, it's as simple as running nix shell --pure in that directory for anyone to reproduce the project environment exactly (security flaws included).

The downsides are:

system dependencies must be added manually, there is no equivalent to apt-get install
package dependencies (pip install, npm install, gem install, go get, etc) not only can't be automated, but are actually much more difficult than system dependencies. Because node and an npm package interact, custom nix code is needed. Understanding nix lang becomes much more of a requirement, and even then it can be quite difficult to get things working (overlays and overrides).
must use lazamars tools for finding older versions, and risk compatibility issues especially with package dependencies
tools like git/npm still read from your $HOME and can cause "well it works on my machine" behaviors
trying to add two binaries with the same name (like two versions of git) does not have an elegant solution, nixpkgs just goes with the last one mentioned in the shell.nix
anything that needs open ssl certificates can have problems
the default shell is bash and there's no way to change it

Some tools like niv have tried to alleviate some downsides. I haven't found any particularly comprehensive, understable or easy to use. I finished my own framework last year to reduce a lot of these problems, but I want to simplify the API before doing a public release.

Many thanks, @jeff-hykin – what I was missing was ‘nix-pinning’. Googling for that, I found ‘Towards reproducibility: pinning Nixpkgs’, which I think I possibly should have been aware of already (is that list of guides new?).

With that, and with a bit of trial-and-error at https://status.nixos.org, I was able to find a channel which contained the specific tool version I needed. That means that I also now understand what Lazamar's tool is doing (each Nix ‘channel’ is a git branch, and the tool identifies the first commit on each channel/branch which has the searched-for tool version – yes?); I was clearly confused about that before.

I completely agree with you on the list of challenges to actual reproducible software. Pinning lets NixPkgs provide step one of a process, and I don't think NixPkgs need aim further up the ladder than that. Even thinking of wrestling with Python package versioning within NixPkgs seems overambitious to me (venvs are everyone's friend!).

As is often the case after one has been cordially pointed to the right page of the FM, I feel a little foolish at not having known the answer already. I don't remember seeing the current contents of learn before, with the list of guides prominently higher than the discussion of Nix pills: if that's always been there, then now I do feel a bit dim; if that's newish, then excellent work whoever reworked the page.

Final reflections:

This does match the use-case I described, and is just about as easy as it should be, once one knows that ‘pinning’ is the right term.
I now see that this present issue is about something different: ‘I'm fine with the channel I'm on, but I want a different version of specific package X’. The above discussion makes more sense now than it did. It's always a little hard to work out, post-enlightenment, why one was confused, but in case it's useful...
The notion of ‘channel’ is not, to me, as clear and obvious as the author of the relevant section of the manual thinks it is. That text describes a channel as ‘a place that contains a set of Nix expressions and a manifest’ – that's a definition in terms of the implementing technology. A definition that's more hand-waving, but possibly more accessible to the naive (eg, me) would be something like ‘a channel is a set of package versions which are expected to work together; the most prominent example of a channel is the set of packages in a “release” such as nixpkgs-21.09-darwin’. Realising that would I believe have given me more traction with the NixPkgs docs at an earlier stage.

This issue is probably as much a documentation issue as a functionality one. The challenge of writing documentation is always the challenge of trying to remember not understanding something. I hope that recording here the nature of my ignorance, while that precious ignorance is fast-fading away, will help advance the issue a little bit.

This issue is probably as much a documentation issue as a functionality one. The challenge of writing documentation is always the challenge of trying to remember not understanding something. I hope that recording here the nature of my ignorance, while that precious ignorance is fast-fading away, will help advance the issue a little bit.

@nxg https://nixos.org/manual/nix/unstable/command-ref/new-cli/nix3-flake.html#flake-inputs

but it is missing using in outputs, like using multiple nixpkgs through specialArgs

There are three ways this can be accomplished:

If there is already a derivation provided by the user's channel, the user need only find its attribute path name. This has gotten easier with nix search, but isn't always reliable. Usually, packages follow the naming convention of appending the version to the package name, but this is not always the case.

This is essentially fixed by search.nixos.org, nix search and a search through the source code in worst case.

If there is not a derivation provided in the user's channel, the user must find a hash that points to the desired derivation version, and use pinning to refer to the package by hash.

This would result in mixing nixpkgs versions which generally works but can lead to major problems due to impureness in some parts of nixos like the graphics driver or every program that uses LD_LIBRARY_PATH, so it generally should be avoided whenever possible.

If nix can't meet that need, then I'm just going to use something like brew, apt, or pacman that has a larger package base

That is wrong, depending on how you count packages, nixpkgs is biggest package repository and closely behind is the AUR.

and then I'll use luavm, rbenv, and asdf to manage versions on an individual basis like before.

Those tools where created because all the classical package manager also did not support every version of an ecosystem. For nix there are already some community projects which try to do this for some ecosystems and that is place where I see those projects. Having them outside of nixpkgs allows them to rapidly make changes, point there focus on the specific ecosystem or even use features which are not suitable for nixpkgs like IFD. Also fully automated imports of an entire ecosystem like pypi are possible which are not suited for nixpkgs.

I read some comments in this thread and the only actionable things except the general well known documentation problem that I identified would require creating a RFC because they changes they require are beyond big.

After 3 years a lot has changed, but I've never stopped thinking about (or stopped running into) this problem. Literally as I'm writing this I'm waiting on a nix-shell building python 3.7 using a nixpkgs commit from 2018. The LD_LIBRARY_PATH issue you mention @SuperSandro2000 is very present as well.

If nix can't meet that need, then I'm just going to use something like brew, apt, or pacman that has a larger package base

That is wrong, depending on how you count packages, nixpkgs is biggest package repository and closely behind is the AUR.

I didn't mean this as an attack but rather a statement of practicality; even right now I'm having to use apt-get and pyenv while I'm unable to get python3.7 working in nix-shell. Doesn't mean I'm not trying to get it working in nix, it just means I can't rely on it. "Bigger" is just the number of times I can brew install/apt-get/pacman something, but can't nix-install the same thing (like the arduino-ide on M1 or the moveit simulator for ROS). Maybe including pacman in the list was unfair as I don't use it all that often.

the only actionable things

I do want to focus this issue on the actionable items instead of complaints.

would require creating a RFC because they changes they require are beyond big

I agree, there is no quick fix, even just making a full roadmap at this moment is impossible because there is so much to do. That said, actions can (and are) being taken, so here's what I've noticed and what I have been exploring.

Action Item 1: Solve the LD_LIBRARY_PATH (permanently)

I have spent a lot of time learning about dynamic linking, glib, ldd, patchelf etc specifically for this problem. I still have a lot left to learn, like getting caught up with how nix-ld works, and if it can be used for this. Based on what I do understand, it seems very much possible that the whole LD_LIBRARY_PATH problem is solvable by constantly updating the RPATH, rather than having lib files that break whenever LD_LIBRARY_PATH is specified. "Possible' doesn't mean without major complications. Instructions that would normally say "add X to your LD_LIBRARY_PATH" will need to be converted into running a patching tool that changes the RPATH for things like .venv files, and if patching existing nixpkgs files, a tool will need to create new derivations with the modified RPATHS. Again, I still don't see anything that makes it impossible and I've been exploring it with some hacked together tools since I often have to mix and match nix commit versions to get packages I need.

I think mixing versions of nixpkgs is necessary to meet all the practical situations of real-life projects, and this is a step towards making it feasible for users who know nothing about dynamic libraries.

If you think this is impossible (literally impossible not just impractical) I'd be happy to hear about it @SuperSandro2000

Action Item 2: Solve "a search through the source code in worst case"

This is essentially fixed

It is not fixed.

Great progress has been made, and I'll discuss actions in a moment, but it is necessary to make it clear searching (not even searching for a version) is still a major problem for nixpkgs. This one I've spent the most time on, and have (what I think is) a good grasp on.

Action Item 2.1: Binary Name search

The smaller aspect of binary name search is one of the problems I believe has been effectively solved. For example, back in 2020 it was extremely difficult for me to find a package for ls because there was no package named ls. At the time I ended up installing lscolors as a makeshift solution because thats all I could find. Now, using search.nixos.org, searching for ls returns coreutils-full as the first result which is what I would've wanted in the first place. A fantastic improvement.

Problem 2.2: Nested Attributes

Now, lets search for another common package AVFoundation for MacOS. It is needed quite frequently for building and does exist on nixpkgs with an exact attribute-name match.

As a user I would probably conclude Nixpkgs just doesn't have it, but lets try searching for it using Github anyways.

None of the first several results give hints as to what to put for nix-env -iA. I'll leave it as an exercise for the reader to figure out the attribute path.

Mechanical, electrical, and civil engineers use apt-get all the time without ever knowing what "grep" is or how to use it, much less vim/git. Yet a common package on nix requires someone to understand advanced file grepping, cloning repos, and/or inner knowledge of how nix structures itself to come up with more clever searches that narrow down the results. I will consider this an issue worth solving as long as this remains the case.

Action Item 2.3: Uniformity

Part of the problem with matching nested attributes (causing the problem above) is the lack of uniformity across packages. There is a recent RFC for flattening Nixpkgs into a uniform structure, that is a major step in the right direction for both maintainability and making it easier to search. That said, it doesn't address nested packages yet.

Right now nixpkgs is still spagetti, uniform packages helps a lot but there's no clearly communicated difference between a package used by nobody and a package that is essential just for stdenv to exist, or a folder with a default.nix that isn't even a package. More confusingly are the packages that are unnecessarily both core and non-core because they're run multiple times using fixpoint recusion. Two months ago I created nixpkgs-skeleton to try and slice through this confusion to help myself and others understand what parts of nixpkgs were "core". Its a far cry from cleaning everything up but its a start.

Action Item 2.4: Indexing & Exploration

To create a searchable interface, there needs to be some way of exploring nixpkgs, and there's at least 2.

Through nix evaluation
Through static anaylsis

Action Item 2.4.1.1: Static Analysis: Tree Sitter

Last month I finally made a static anaylsis breakthrough getting the nix-tree-sitter running (written by someone else) and using it to create a nix-bundler system (simlar to how javascript bundlers work). This is the basis for extracting information about nested package attributes, but has some common static analysis problems I'll mention in 2.4.1.3.

Action Item 2.4.1.2: Static Analysis: RNix

One week after that I found out about the rnix-parser which is a more comprehensive parser written in rust. Its what is being used for automatic re-writes for making the nixpkgs structure flat.

Action Item 2.4.1.3: Static Analysis: Problems

There's at least two problems that hurt static analysis significantly.

The with keyword (syntactic sugar) ruins static analysis. Its impossible to know the source of a variable inside mutliple with statements, and even inside only one with statement it is problematic. I would like to eventually propose deprecating with, similar to how Javascript deprecated their own with keyword for similar reasons. Actually it looks like someone has beat me to it.
Highly dynamic fields, such as packages that accept their version number as an argument, and then interpolate it into a string are difficult to analyze.

Action Item 2.4.2: Evaluation as Analysis

The try-to-evaluate functionality of nix is somewhat limited and difficult to use. For example, back when I first started using nix-env queries to build a search index, there were certain packages on certain nixpkgs commits that fail on darwin, causing the entire query to fail, meaning no data about any packages after that are returned. Instead there would need to be 80,000 individual queries just to make sure that failures of one package didn't make it impossible to get information about other packages.

More recently I've attempted to write an expression for recursively exploring packages, to drill down into those attributes. However there is a similar issue, packages fail to evaluate, some failures cannot be caught, so I had make a fork and edit out all the cases that would throw uncatchable errors. Having a mode for exploring nixpkgs that allows for everything to be caught would be quite helpful.

Another aspect is that because of the lazy evaluation try-eval often needs to be combined with deepSeq which often doesn't work for getting around the problem. As I learned recently, there is a makeshift work around specifically for derivations recurseForDerivations which I'll be checking out next time I'm working on this task.

Evaluation is more comprehensive since it accounts for dynamic attribtues, but this can also cause problems when evaulating packages for non-supported systems. For example, trying to get the version number of a cuda package on a non-cuda machine.

I worked a bit on a propsal for flakes to have a meta or static tag that was evaluatable on every machine. But later settled that the flake.lock format would be better suited for storing indexable data.

Action Item 3: Version Publications

Those things above are mostly just working on package discovery, even without versions, but for versions, compatibility checks etc, final action is working on a system of simulaniously distributed and centralized packages based of this. Where packages can be published and added to a registry independent of whether or not they are integrated into nixpkgs. This will require a server with authentication listening for publish commands, as well as its own search system. I can't say I'm close, but I have been writing code for this for a couple years now.

In terms of actionable steps, getting flakes nailed down will be extremely helpful for this case. Both with the flake-lock being great equivlent to something similar to a package.json, and with the inputs more easily allowing pulling in code from sources other than nixpkgs.

TLDR

There aren't many messages, exciting demos/changes, but a lot of actions have been done and are worth doing towards solving this issue. I might complain a lot, but its because I want nixpkgs to be all it can be instead of what it is.

Action Item 2.1: Binary Name search

The smaller aspect of binary name search is one of the problems I believe has been effectively solved. For example, back in 2020 it was extremely difficult for me to find a package for ls because there was no package named ls. At the time I ended up installing lscolors as a makeshift solution because thats all I could find. Now, using search.nixos.org, searching for ls returns coreutils-full as the first result which is what I would've wanted in the first place. A fantastic improvement.

That's probably pretty easy to fix when we have replaced command-not-found with nikx-locate, someone would just need to write some converter to elastic and expose i on the website.

Now, lets search for another common package AVFoundation for MacOS. It is needed quite frequently for building and does exist on nixpkgs with an exact attribute-name match.

There is probably a recurseInto somewhere missing or we are not evaluating those parts on linux, so those things are missed. Not fully sure.

Don't really have something to say on the other things right now.

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/best-practice-for-pinning-version-of-individual-packages/6194/7

devbox solves this problem by letting you specify package versions within a json file, it's a great nix wrapper for mere mortals.

So I think there's still a problem worth solving here, but it's not really a problem with devbox as much as it is that this issue thread is an upstream issue for devbox.

I've been working on/using a devbox-like tool ("virkshop" the virtual workshop). I want to provide similar features like versions but both myself and devbox can't AFAIK because:

It's not possible to programatically find/list all available versions (please correct me if devbox has invented their own static-analysis or dynamic-analysis tool that solves this; that kind of solution would close this whole issue thread)
Even when you find a package, pinning the nixpkgs version (which devbox does let you do) will cause stuff like the LD_LIBRARY_PATH problem that SuperSandro2000 brought up a few comments ago.

All that said; I would still highly recommend trying out devbox in the meantime! Once this issue is fixed devbox should simply work better than ever before. I didn't know about it till it was mentioned here (thank you @dudicoco), and I'm pretty excited to try it out and see if it can fully replace the tool I've been working on.

So there's still a problem, but it's not really with devbox moreso this issue thread is an upstream issue for devbox.

I've been working on/using a devbox-like tool ("virkshop" the virtual workshop). I want to provide similar features like versions but both myself and devbox can't because:

It's not possible to programatically find/list all available versions (please correct me if devbox has invented their own static-analysis or dynamic-analysis tool that solves this; that kind of solution would close this whole issue thread)

Even when you find a package, pinning the nixpkgs version (which devbox does let you do) will cause stuff like the LD_LIBRARY_PATH problem that SuperSandro2000 brought up a few comments ago.

All that said; I would still highly recommend trying out devbox in the meantime! Once this issue is fixed devbox should simply work better than ever before. I didn't know about it till it was mentioned here (thank you @dudicoco), and I'm pretty excited to try it out and see if it can fully replace the tool I've been working on.

@jeff-hykin devbox does support searching for available package versions within its CLI: https://www.jetpack.io/devbox/docs/cli_reference/devbox_search/

Answer Update: How to find and install a specific version

Note: Devbox is not designed for this, but it is the most exhaustive solution at the moment. Devbox is a framework, it's not really designed to work "with nix", it moreso uses nix under the hood. This tutorial is only how to use it to find/install nix packages versions.

# install devbox
curl -fsSL https://get.jetpack.io/devbox | bash       # can also use nix: nix-env -iA nixpkgs.devbox

# exhaustive search
devbox search nodejs --show-all

# nodejs  (20.3.1, 20.2.0, 20.0.0, 19.9.0, 19.8.1, 19.8.0, 19.7.0, 19.6.1, 19.6.0, 19.5.0, 19.4.0, 19.2.0, 19.1.0, 19.0.1, 19.0.0, 18.16.1, 18.16.0, 18.15.0, 18.14.2, 18.14.1, 18.14.0, 18.13.0, 18.12.1, 18.12.0, 18.10.0, 18.9.1, 18.9.0, 18.8.0, 18.7.0, 18.6.0, 18.5.0, 18.4.0, 18.3.0, 18.2.0, 18.1.0, 18.0.0, 17.9.0, 17.8.0, 17.7.2, 17.5.0, 17.4.0, 17.3.1, 17.3.0, 17.2.0, 17.1.0, 17.0.1, 16.20.1, 16.20.0, 16.19.1, 16.19.0, 16.18.1, 16.18.0, 16.17.1, 16.17.0, 16.16.0, 16.15.1, 16.15.0, 16.14.2, 16.14.0, 16.13.2, 16.13.1, 16.13.0, 16.12.0, 16.10.0, 16.9.1, 16.9.0, 16.8.0, 16.7.0, 16.6.2, 16.6.1, 16.6.0, 16.5.0, 16.4.2, 16.4.1, 16.4.0, 16.3.0, 16.2.0, 16.1.0, 16.0.0, 15.14.0, 15.13.0, 15.12.0, 15.11.0, 15.10.0, 15.9.0, 15.8.0, 15.7.0, 15.6.0, 15.5.1, 15.5.0, 15.4.0, 15.3.0, 15.2.1, 15.2.0, 15.1.0, 15.0.1, 14.21.3, 14.21.2, 14.21.1, 14.21.0, 14.20.1, 14.20.0, 14.19.3, 14.19.2, 14.19.1, 14.19.0, 14.18.3, 14.18.2, 14.18.1, 14.18.0, 14.17.6, 14.17.5, 14.17.4, 14.17.3, 14.17.2, 14.17.1, 14.17.0, 14.16.1, 14.16.0, 14.15.5, 14.15.4, 14.15.3, 14.15.1, 14.15.0, 14.14.0, 14.9.0, 14.8.0, 14.7.0, 14.6.0, 14.5.0, 14.4.0, 13.14.0, 12.22.12, 12.22.11, 12.22.10, 12.22.9, 12.22.8, 12.22.7, 12.22.6, 12.22.5, 12.22.4, 12.22.3, 12.22.2, 12.22.1, 12.22.0, 12.21.0, 12.20.2, 12.20.1, 12.20.0, 12.19.1, 12.19.0, 12.18.4, 12.18.3, 12.18.2, 12.18.1, 12.18.0, 10.24.1, 10.24.0, 10.23.3, 10.23.2, 10.23.1)
# nodejs-slim  (20.3.1, 20.2.0, 20.0.0, 19.9.0, 19.8.1, 19.8.0, 19.7.0, 19.6.1, 19.6.0, 19.5.0, 19.4.0, 19.2.0, 19.1.0, 19.0.1, 19.0.0, 18.16.1, 18.16.0, 18.15.0, 18.14.2, 18.14.1, 18.14.0, 18.13.0, 18.12.1, 18.12.0, 18.10.0, 18.9.1, 18.9.0, 18.8.0, 18.7.0, 18.6.0, 18.5.0, 18.4.0, 18.3.0, 18.2.0, 18.1.0, 18.0.0, 17.9.0, 17.8.0, 17.7.2, 17.5.0, 17.4.0, 17.3.1, 17.3.0, 17.2.0, 17.1.0, 17.0.1, 16.20.1, 16.20.0, 16.19.1, 16.19.0, 16.18.1, 16.18.0, 16.17.1, 16.17.0, 16.16.0, 16.15.1, 16.15.0, 16.14.2, 16.14.0, 16.13.2, 16.13.1, 16.13.0, 16.12.0, 16.10.0, 16.9.1, 16.9.0, 16.8.0, 16.7.0, 16.6.2, 16.6.1, 16.6.0, 16.5.0, 16.4.2, 16.4.1, 16.4.0, 16.3.0, 16.2.0, 16.1.0, 16.0.0, 15.14.0, 15.13.0, 15.12.0, 15.11.0, 15.10.0, 15.9.0, 15.8.0, 15.7.0, 15.6.0, 15.5.1, 15.5.0, 15.4.0, 15.3.0, 15.2.1, 15.2.0, 15.1.0, 15.0.1, 14.21.3, 14.21.2, 14.21.1, 14.21.0, 14.20.1, 14.20.0, 14.19.3, 14.19.2, 14.19.1, 14.19.0, 14.18.3, 14.18.2, 14.18.1, 14.18.0, 14.17.6, 14.17.5, 14.17.4, 14.17.3, 14.17.2, 14.17.1, 14.17.0, 14.16.1, 14.16.0, 14.15.5, 14.15.4, 14.15.3, 14.15.1, 14.15.0, 14.14.0, 14.9.0, 14.8.0, 14.7.0, 14.6.0, 14.5.0, 14.4.0, 13.14.0, 12.22.12, 12.22.11, 12.22.10, 12.22.9, 12.22.8, 12.22.7, 12.22.6, 12.22.5, 12.22.4, 12.22.3, 12.22.2, 12.22.1, 12.22.0, 12.21.0, 12.20.2, 12.20.1, 12.20.0, 12.19.1, 12.19.0, 12.18.4, 12.18.3, 12.18.2, 12.18.1, 12.18.0, 10.24.1, 10.24.0, 10.23.3, 10.23.2, 10.23.1)
# nodejs_latest  (20.0.0, 19.9.0, 19.8.1, 19.8.0, 19.7.0, 19.6.1, 19.6.0, 19.5.0, 19.4.0, 19.2.0, 19.1.0, 19.0.1, 19.0.0, 18.10.0, 18.9.1, 18.9.0, 18.8.0, 18.7.0, 18.6.0, 18.5.0, 18.4.0, 18.3.0, 18.2.0, 18.1.0, 18.0.0, 16.14.2, 16.14.0, 16.13.2, 16.13.1, 16.13.0, 16.12.0, 16.10.0, 16.9.1, 16.9.0, 16.8.0, 16.7.0, 16.6.2, 16.6.1, 16.6.0, 16.5.0, 16.4.2, 16.4.1, 16.4.0, 16.3.0, 16.2.0, 16.1.0, 16.0.0, 15.14.0, 15.13.0, 15.12.0, 15.11.0, 15.10.0, 15.9.0, 15.8.0, 15.7.0, 15.6.0, 15.5.1, 15.5.0, 15.4.0, 15.3.0, 15.2.1, 15.2.0, 15.1.0, 15.0.1, 14.9.0, 14.8.0, 14.7.0, 14.6.0, 14.5.0, 14.4.0)
# nodejs-slim_latest  (20.0.0, 19.9.0, 19.8.1, 19.8.0, 19.7.0, 19.6.1, 19.6.0, 19.5.0, 19.4.0, 19.2.0, 19.1.0, 19.0.1, 19.0.0, 18.10.0, 18.9.1, 18.9.0, 18.8.0, 18.7.0, 18.6.0, 18.5.0, 18.4.0, 18.3.0, 18.2.0, 18.1.0, 18.0.0, 16.14.2, 16.14.0, 16.13.2, 16.13.1, 16.13.0, 16.12.0, 16.10.0, 16.9.1, 16.9.0, 16.8.0, 16.7.0, 16.6.2, 16.6.1, 16.6.0, 16.5.0, 16.4.2, 16.4.1, 16.4.0, 16.3.0, 16.2.0, 16.1.0, 16.0.0, 15.14.0, 15.13.0, 15.12.0, 15.11.0, 15.10.0, 15.9.0, 15.8.0, 15.7.0, 15.6.0, 15.5.1, 15.5.0, 15.4.0, 15.3.0, 15.2.1, 15.2.0, 15.1.0, 15.0.1, 14.9.0, 14.8.0, 14.7.0, 14.6.0, 14.5.0, 14.4.0)
# nodejs-16_x-openssl_1_1  (16.19.0, 16.18.1, 16.18.0, 16.17.1, 16.17.0)

# for your whole system
devbox global add nodejs@19.9.0
devbox global install

# for your specific project/folder
devbox init
devbox add nodejs@19.9.0
devbox shell # mimicing nix-shell

To get the nixpkg hash of the package (so it can be installed without devbox, or used in a shell.nix) you'll have to open up your devbox.lock

If global installing, check cat $XDG_DATA_HOME/devbox/global/default/devbox.lock and/or cat ~/.local/share/devbox/global/default/devbox.lock
if local then it will just be in the local ./devbox.lock

Caveats / Notes:

"Search api lacks x86-darwin info" is in a comment and I imagine there are many other such caveats because of this issue (packages don't have a consistent way of listing their versions within nixpkgs)
Says "experimental" on the Devbox site (at least at the moment, July 24th 2023)
The LD_LIBRARY_PATH problem mentioned a few comments above is still going to be present

Thanks again for the update @dudicoco. I'll have to spend more time reading their source code to figure out how they're doing it (it is still pretty tough for me to imagine they perfectly/ideally solved version search). But this is definitely the best most-comprehensive version search for nixpkgs I've seen, so they've got to be doing something right!

Update: The Version-Search Results are now Updated Daily! (In sync with nixpkgs unstable)

From the guys at devbox: https://www.nixhub.io/ (Which is also powers their CLI search in the comment above). As of today; as soon as a version is available in nixpkgs unstable; within 24 hours it'll show up on those search tools.

Note: There are still search caveats, like Mac M1, but this is still a huge amount of progress on this issue and all the credit goes to the devbox team. They also have a great article here talking about how they canoncialize names and versions since Nixpkgs still doesn't try to.

I met with 3 devbox engineers last month and we've been working together to 1. spearhead getting all the missing versions cataloged and 2. get all the code to be open source. Some nice incremental achievement is finished a script that can fully exhaustively brute force search all attributes (which AFAIK not even hydra or nix-env is capable of). The script/approach is too slow for daily updates, but it will help us figure out why nix-env misses packages. If we can figure that out and fix it, then we will have an entirely clear path for indexing every version that's ever existed on nixpkgs in a efficient way, even super old nicpkgs. From there it'll be a matter of building the final bits of infrastructure, open sourcing it, and getting it to be part of the normal nix-system/nix-search.

Update (Nov 2023): The devbox guys still haven't open sourced the indexing tool which is sad, but I haven't pinged them about it.

LD_PATH is still a problem, but honestly that problem belongs in a different issue.

We now all know about devbox but could we not turn this thread into an ad for it? That would be nice.

@SuperSandro2000 I agree and I see why it comes off that way.

There were no complaints when I posted a tutorial for Lazamar's website, probably because their website is designed to work with nix, and its only job is version search. (Clearly Lazamar had no ulterior motive)

I do not have a single project that uses devbox; I have never indented to advertise anything other than a how to find/install specific versions of nix packages. The devbox tutorial had a lot of detail because (unfortunately) Devbox is a bulky framework, and devbox's version search isn't designed to work with nix. Getting a nixpkgs commit hash out of Devbox is an extreme pain compared to Lazamar's site. It took me quite a while to find how to get the nixpkgs hash, so I wanted to make a tutorial for the people who cared about that.

I agree that^ difference is worth clarifying so I added a disclaimer at the top, and tried to make the nix-without-devbox usage more prominent. Let me know if you think it can be better @SuperSandro2000

I'll also do another answer summary to show all the options.

TLDR; How to find and install a specific version

Search for a version here https://history.nix-packages.com/ (the successor to Lazamar's website; RikudoeSage's site!) and start at Step 1 of this tutorial
If the version isn't on there, then try finding it using devbox tutorial here.

I'd just add that the instructions are there for each version if you click into the version's detail.

I think I have found a "native" solution to the problem of installing old versions (at least for my own use case). In summary, to install an old package with nix,

locate a cached old version of the package on hydra: https://hydra.nixos.org
install the cached closure via nix profile install or builtins.fetchClosure

A detailed write-up is here: https://github.com/bryango/cheznix/blob/master/doc/cached-versions.md

NixOS / nixpkgs

There isn't a clear canonical way to refer to a specific package version. #93327

Context

Nix Design

Nix Usage

Problem

Update for this Issue

Action Item 1: Solve the LD_LIBRARY_PATH (permanently)

Action Item 2: Solve "a search through the source code in worst case"

Action Item 2.1: Binary Name search

Problem 2.2: Nested Attributes

Action Item 2.3: Uniformity

Action Item 2.4: Indexing & Exploration

Action Item 2.4.1.1: Static Analysis: Tree Sitter

Action Item 2.4.1.2: Static Analysis: RNix

Action Item 2.4.1.3: Static Analysis: Problems

Action Item 2.4.2: Evaluation as Analysis

Action Item 3: Version Publications

TLDR

Action Item 2.1: Binary Name search

Answer Update: How to find and install a specific version

Caveats / Notes:

Update: The Version-Search Results are now Updated Daily! (In sync with nixpkgs unstable)

TLDR; How to find and install a specific version