NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
16.48k stars 12.97k forks source link

unstable part of version instead of pname #68518

Open jtojnar opened 4 years ago

jtojnar commented 4 years ago

As discussed in https://github.com/repology/repology/issues/854#issuecomment-530450656, in many packages we currently have unstable a part of version, rather than pname suggested in https://nixos.org/nixpkgs/manual/#sec-package-naming. I know I am guilty of promulgating this and we should probably fix that before switching Repology to pname & version.

$ rg version.+=.+unstable pkgs
pkgs/servers/echoip/default.nix
pkgs/servers/demoit/default.nix
pkgs/games/gnome-hexgl/default.nix
pkgs/games/warmux/default.nix
pkgs/misc/lguf-brightness/default.nix
pkgs/games/gscrabble/default.nix
pkgs/misc/uq/default.nix
pkgs/misc/sound-of-sorting/default.nix
pkgs/games/frogatto/default.nix
pkgs/games/frogatto/engine.nix
pkgs/games/frogatto/data.nix
pkgs/servers/mumsi/default.nix
pkgs/servers/webmetro/default.nix
pkgs/development/beam-modules/hex-registry-snapshot.nix
pkgs/development/lua-modules/overrides.nix
pkgs/desktops/surf-display/default.nix
pkgs/games/qgo/default.nix
pkgs/games/devilutionx/default.nix
pkgs/games/assaultcube/default.nix
pkgs/data/fonts/gelasio/default.nix
pkgs/data/fonts/noto-fonts/tools.nix
pkgs/data/fonts/lalezar-fonts/default.nix
pkgs/data/fonts/ir-standard-fonts/default.nix
pkgs/data/fonts/xkcd-font/default.nix
pkgs/servers/monitoring/prometheus/json-exporter.nix
pkgs/development/compilers/crystal/crystal2nix.nix
pkgs/development/compilers/carp/default.nix
pkgs/development/compilers/z88dk/default.nix
pkgs/development/compilers/gerbil/unstable.nix
pkgs/development/compilers/openspin/default.nix
pkgs/development/compilers/qbe/default.nix
pkgs/development/compilers/gambit/unstable.nix
pkgs/development/misc/rappel/default.nix
pkgs/development/libraries/dee/default.nix
pkgs/development/libraries/libgumath/default.nix
pkgs/development/libraries/libunity/default.nix
pkgs/development/libraries/libxnd/default.nix
pkgs/development/python-modules/tess/default.nix
pkgs/development/tools/rdocker/default.nix
pkgs/development/tools/fusee-launcher/default.nix
pkgs/development/tools/quicktemplate/default.nix
pkgs/development/tools/ofono-phonesim/default.nix
pkgs/development/libraries/libelfin/default.nix
pkgs/development/tools/osslsigncode/default.nix
pkgs/development/libraries/mutest/default.nix
pkgs/development/libraries/libndtypes/default.nix
pkgs/development/tools/statik/default.nix
pkgs/development/tools/go-outline/default.nix
pkgs/development/libraries/multipart-parser-c/default.nix
pkgs/development/tools/yaml2json/default.nix
pkgs/development/libraries/mumlib/default.nix
pkgs/development/libraries/yubico-pam/default.nix
pkgs/development/libraries/coin3d/default.nix
pkgs/development/python-modules/nxt-python/default.nix
pkgs/development/ocaml-modules/ppx_tools/default.nix
pkgs/development/libraries/soxt/default.nix
pkgs/development/libraries/ticpp/default.nix
pkgs/development/libraries/rapidcheck/default.nix
pkgs/development/libraries/g2o/default.nix
pkgs/development/python-modules/nmigen-boards/default.nix
pkgs/development/python-modules/pybluez/default.nix
pkgs/development/libraries/beignet/default.nix
pkgs/development/ocaml-modules/ocaml-libvirt/default.nix
pkgs/development/python-modules/ptable/default.nix
pkgs/development/tools/dep2nix/default.nix
pkgs/development/libraries/gthree/default.nix
pkgs/development/python-modules/nmigen/default.nix
pkgs/development/python-modules/glasgow/default.nix
pkgs/development/libraries/libykclient/default.nix
pkgs/tools/backup/iceshelf/default.nix
pkgs/tools/backup/easysnap/default.nix
pkgs/tools/backup/diskrsync/default.nix
pkgs/development/tools/vgo2nix/default.nix
pkgs/development/tools/easyjson/default.nix
pkgs/development/python-modules/google_cloud_testutils/default.nix
pkgs/development/python-modules/backports_shutil_get_terminal_size/default.nix
pkgs/tools/networking/openconnect_pa/default.nix
pkgs/development/python-modules/pytest-ordering/default.nix
pkgs/development/python-modules/pymaging_png/default.nix
pkgs/development/python-modules/nbmerge/default.nix
pkgs/tools/package-management/cargo-outdated/default.nix
pkgs/development/python-modules/fx2/default.nix
pkgs/tools/misc/vdirsyncer/default.nix
pkgs/tools/misc/ltunify/default.nix
pkgs/tools/misc/zabbixctl/default.nix
pkgs/tools/misc/gif-for-cli/default.nix
pkgs/development/python-modules/python-unshare/default.nix
pkgs/tools/X11/xcape/default.nix
pkgs/tools/misc/cht.sh/default.nix
pkgs/development/python-modules/pyfantom/default.nix
pkgs/tools/misc/dpt-rp1-py/default.nix
pkgs/tools/misc/fffuu/default.nix
pkgs/tools/misc/pandoc-imagine/default.nix
pkgs/tools/misc/uhubctl/default.nix
pkgs/tools/graphics/yaxg/default.nix
pkgs/tools/graphics/svgbob/default.nix
pkgs/os-specific/linux/fscrypt/default.nix
pkgs/tools/security/ripasso/cursive.nix
pkgs/tools/security/b2sum/default.nix
pkgs/os-specific/linux/facetimehd/default.nix
pkgs/os-specific/linux/anbox/default.nix
pkgs/applications/window-managers/yabar/unstable.nix
pkgs/applications/window-managers/i3/pystatus.nix
pkgs/applications/window-managers/i3/layout-manager.nix
pkgs/applications/window-managers/kbdd/default.nix
pkgs/applications/window-managers/i3/blocks.nix
pkgs/os-specific/linux/ledger-udev-rules/default.nix
pkgs/applications/networking/flent/http-getter.nix
pkgs/applications/networking/ps2client/default.nix
pkgs/applications/misc/antfs-cli/default.nix
pkgs/applications/virtualization/virt-top/default.nix
pkgs/applications/misc/qt-box-editor/default.nix
pkgs/applications/editors/amp/default.nix
pkgs/applications/misc/gpscorrelate/default.nix
pkgs/applications/misc/web-media-controller/default.nix
pkgs/applications/misc/tilix/default.nix
pkgs/applications/misc/tdrop/default.nix
pkgs/development/python-modules/pymaging/default.nix
pkgs/applications/misc/perkeep/default.nix
pkgs/development/python-modules/pytricia/default.nix
pkgs/applications/misc/cataract/unstable.nix
pkgs/applications/misc/kjv/default.nix
pkgs/applications/misc/pbpst/default.nix
pkgs/applications/misc/slstatus/default.nix
pkgs/applications/misc/autospotting/default.nix
pkgs/applications/misc/lifelines/default.nix
pkgs/applications/misc/ssocr/default.nix
pkgs/applications/misc/bicon/default.nix
pkgs/applications/misc/glom/default.nix
pkgs/applications/office/planner/default.nix
pkgs/applications/office/bookworm/default.nix
pkgs/applications/graphics/ufraw/default.nix
pkgs/applications/video/mjpg-streamer/default.nix
pkgs/applications/audio/rhvoice/default.nix
pkgs/applications/audio/speech-denoiser/default.nix
pkgs/applications/audio/musly/default.nix
pkgs/servers/sql/postgresql/ext/pgjwt.nix
pkgs/development/tools/build-managers/redo-apenwarr/default.nix
pkgs/development/tools/misc/blackmagic/default.nix
pkgs/development/tools/misc/universal-ctags/default.nix
pkgs/development/tools/analysis/panopticon/default.nix
pkgs/tools/typesetting/tex/latexrun/default.nix
pkgs/desktops/pantheon/desktop/gala/default.nix
pkgs/desktops/pantheon/artwork/elementary-redacted-script/default.nix
pkgs/desktops/gnome-3/core/vino/default.nix
pkgs/desktops/gnome-3/extensions/window-corner-preview/default.nix
pkgs/desktops/gnome-3/extensions/tilingnome/default.nix
pkgs/desktops/gnome-3/extensions/timepp/default.nix
pkgs/desktops/gnome-3/extensions/sound-output-device-chooser/default.nix
pkgs/desktops/gnome-3/extensions/mediaplayer/default.nix
pkgs/desktops/gnome-3/extensions/caffeine/default.nix
pkgs/applications/window-managers/xmonad/log-applet/default.nix
pkgs/applications/networking/remote/x2goclient/default.nix
pkgs/applications/networking/instant-messengers/chatterino2/default.nix
pkgs/applications/version-management/git-and-tools/git-reparent/default.nix
pkgs/applications/version-management/git-and-tools/gitstatus/default.nix
pkgs/applications/science/biology/poretools/default.nix
pkgs/applications/science/electronics/fped/default.nix
pkgs/applications/science/electronics/verilog/default.nix
pkgs/applications/graphics/gimp/plugins/default.nix
pkgs/applications/audio/deadbeef/plugins/lyricbar.nix
pkgs/applications/audio/csound/csound-manual/default.nix
pkgs/applications/networking/instant-messengers/pidgin-plugins/pidgin-opensteamworks/default.nix
pkgs/applications/networking/instant-messengers/pidgin-plugins/purple-discord/default.nix

The documentation should probably be clarified re this. cc @volth @worldofpeace

worldofpeace commented 4 years ago

But even with this scheme pname won't be the upstream name. Is the rule even desirable? What does repology do to consider package version which are commits off the source repo?

jtojnar commented 4 years ago

Repology can mark packages ending with unstable as ignored (light gray).

jtojnar commented 4 years ago

The main issue with where to place unstable probably comes from the fact that we use it for two different things:

Nix makes it hard to tell which is which since it does not know any blessed attributes other than name, and parseDrvName only recognizes the first case.

worldofpeace commented 4 years ago

@jtojnar and I also talked about this on IRC #nixos-dev

(note: when I say "attributes" I meant attrPath in all-packages.nix)

ryantm commented 4 years ago

@volth made a good summary comment about this https://github.com/NixOS/nixpkgs/issues/68531#issuecomment-533760929

stale[bot] commented 4 years ago

Thank you for your contributions. This has been automatically marked as stale because it has had no activity for 180 days. If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity. Here are suggestions that might help resolve this more quickly:

  1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
  2. Ask on the NixOS Discourse. 3. Ask on the #nixos channel on irc.freenode.net.
jtojnar commented 4 years ago

Still unresolved.

stale[bot] commented 3 years ago

I marked this as stale due to inactivity. → More info

davidak commented 3 years ago

The initial issue lists 164 affected issues. Now it seem to be 395.

[davidak@gaming:~/code/nixpkgs]$ egrep -R "version.+=.+unstable" pkgs/ | wc -l
395

I will look into it.

AndersonTorres commented 3 years ago

About this issue. I am confused.

Sometimes we have a software like DWM, with a numbered official version and a git version both maintained by us. They differ by pname, not merely by version.

On the other hand, we use version = unstable-${fancy-date} in other cases like this.

What convention we should follow, after all?

davidak commented 3 years ago

it stresses me to see that this is still not solved. we have rules, but prs following the rules are rejected because the rules are no good

nixpkgs is a mess and i can't clean it up :/

maybe the RFC process could help us to decide on a rule

there are arguments against unstable in pname (nix-env breakes) if unstable is better a prefix or suffx to version might be subjective, so we might just vote what feels better if we don't find objective arguments... maybe we can remove unstable alltogether because YYYY-MM-DD is clearly not an official release (is there a case where it is? https://calver.org/ uses dots)

anyone want to bring this forward?

AndersonTorres commented 3 years ago

nixpkgs is a mess and i can't clean it up :/

It will be a sprintable job after all

there are arguments against unstable in pname (nix-env breakes)

And, thinking about the repology issue, unstable in pname is treated as pname, so that 'blob-unstable' is different from 'blob'.

if unstable is better a prefix or suffx to version might be subjective, so we might just vote what feels better if we don't find objective arguments...

Prefix is better. It is an important information, and putting it before the version numbering makes it clear.

maybe we can remove unstable alltogether because YYYY-MM-DD is clearly not an official release (is there a case where it is? https://calver.org/ uses dots)

youtube-dl uses dots too, but I think we can't blindly suppose no one will write its releases that way. Putting unstable-YYYY-MM-DD makes it unambiguous.

anyone want to bring this forward?

I am merely converting everything I am touching to that format, and applying some manual cleanups

davidak commented 3 years ago

So one step forward would be to create an RFC that suggests to use version = "unstable-YYYY-MM-DD"; and see if it will be accepted. Add a conclusion from all the discussions here.

@AndersonTorres would you like to do that?

AndersonTorres commented 3 years ago

My main concern here is about the parseDrvName issue. The way they are written now, version can't include unstable on it.

AndersonTorres commented 3 years ago

@davidak I would like it. What can I do?

jtojnar commented 3 years ago

it stresses me to see that this is still not solved. we have rules, but prs following the rules are rejected because the rules are no good

I would say that for now we should just follow the rules, even if suboptimal. Rejecting PRs because they follow rules we do not like is terrible for the submitter.

there are arguments against unstable in pname (nix-env breakes)

nix-env does not care whether unstable tag is at the end of pname or at the beginning of version, it takes the resulting name and runs parseDrvName on it.

The argument, then, is that putting it in version is wrong because pname and version attributes will no longer match parseDrvName name.

And since nix-env uses parseDrvName, even if we moved the tag to version, Repology would still not be able to make use of it. I opened https://github.com/NixOS/nix/pull/4463 to make nix-env aware of the widely used convention of separate pname and version attributes but it has not been well received by the powers that be.

if unstable is better a prefix or suffx to version might be subjective, so we might just vote what feels better if we don't find objective arguments...

The alternative of moving the tag to the end of version as described in https://github.com/NixOS/nixpkgs/pull/100833#issuecomment-712541325 makes a trade-offs that need to be carefully considered. It would be okay for both parseDrvName and clarity. Disadvantage would be that nix-env would not update from unstable version to stable after stabilization because year would be almost certainly greater than whatever stable major version. But that is an issue with the current rule as well – even worse, actually, since nix-env will consider them two different packages and will not update even with --always (which should IMO be the default).

because YYYY-MM-DD is clearly not an official release (is there a case where it is? https://calver.org/ uses dots)

There are a few. Feel free to grep nixpkgs.


And, thinking about the repology issue, unstable in pname is treated as pname, so that 'blob-unstable' is different from 'blob'.

Repology can filter that out but yeah, it is not ideal.

if unstable is better a prefix or suffx to version might be subjective, so we might just vote what feels better if we don't find objective arguments...

Prefix is better. It is an important information, and putting it before the version numbering makes it clear.

I would say version is short enough that it will be clear either way. Putting it after has benefit of attributes being consistent with parseDrvName.

maybe we can remove unstable alltogether because YYYY-MM-DD is clearly not an official release (is there a case where it is? calver.org uses dots)

youtube-dl uses dots too, but I think we can't blindly suppose no one will write its releases that way. Putting unstable-YYYY-MM-DD makes it unambiguous.

Yeah, there are several such packages in nixpkgs.

anyone want to bring this forward?

I am merely converting everything I am touching to that format, and applying some manual cleanups

I would say any changes until we decide on the rule are premature.

So one step forward would be to create an RFC that suggests to use version = "unstable-YYYY-MM-DD"; and see if it will be accepted. Add a conclusion from all the discussions here.

My main concern here is about the parseDrvName issue. The way they are written now, version can't include unstable on it.

That is Eelco’s concern as well and he is very adamant about it.

I would like it. What can I do?

The first step would probably be formulating why are the current rules not good enough. Then we can compile list of alternatives and compare them against the criteria. It would be especially useful to look at what other distros do. IIRC, Arch did something like $lastStableVersion-git+$commitHash which would solve the parseDrvName issue and upgrades between stable and temporarily unstable versions – but not upgrades between unstable versions without --always since commit hashes are not monotonous.

7c6f434c commented 3 years ago

Disadvantage would be that nix-env would not update from unstable version to stable after stabilization because year would be almost certainly greater than whatever stable major version. But that is an issue with the current rule as well – even worse, actually, since nix-env will consider them two different packages and will not update even with --always (which should IMO be the default).

(That's why I sometimes give up and insert a zero component before the year so that the ordering is right)

The first step would probably be formulating why are the current rules not good enough. Then we can compile list of alternatives and compare them against the criteria. It would be especially useful to look at what other distros do. IIRC, Arch did something like $lastStableVersion-git+$commitHash which would solve the parseDrvName issue and upgrades between stable and temporarily unstable versions – but not upgrades between unstable versions without --always since commit hashes are not monotonous.

$lastStableVersion-unstable-$date-$hash would be so nice to have agreed…

AndersonTorres commented 3 years ago

@7c6f434c

$lastStableVersion-unstable-$date-$hash would be so nice to have agreed…

No, no, no. ${last-stable-version}-unstable-${date in yyyy-mm-dd format} should be good enough. And, for the sake of unambiguity, a project without clearcut releases should have a default last-stable-version := 0.0.0. Hashes are low-level info. For us non-machines, dates are way more useful than a cryptical soup of letters generated by a machine.

The only problematic thing would be if some project released many modifications in Unstable/Trunk/Master/Main/WTH branch in a single day. and therefore date would not be unambiguous. But we can use the last commit of yesterday if we are to be overly cautious.

I liked that idea! What you all think?


@jtojnar

The first step would probably be formulating why are the current rules not good enough.

I would say ambiguity is the issue here. Many possibilities and no uniformity.

Then we can compile list of alternatives and compare them against the criteria.

My current stance is ${last-stable-version}-unstable-${date in yyyy-mm-dd format}. I think it is a good compromise between not changing parseDrvName and keeping useful info about version.

Arch did something like $lastStableVersion-git+$commitHash

IIANW, it is mostly from AUR. Anyway I think the model above captures the intent well enough.

7c6f434c commented 3 years ago

For us non-machines, dates are way more useful than a cryptical soup of letters generated by a machine.

Yes, that's why date should be before the hash! But I do not insist on hash and will support the no-hash version if you write an RFC or a PR to the manual

nixos-discourse commented 3 years ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/brainstorming-for-rfc-pname-and-version/12873/1

figsoda commented 2 years ago

I like the idea of 0.0.0-unstable.2021-09-23 or 0.0.0-git.2021-09-23, which follows semantic versioning

AndersonTorres commented 2 years ago

I will upload a RFC shortly.

AndersonTorres commented 2 years ago

As promised:

https://github.com/NixOS/rfcs/pull/107

stale[bot] commented 2 years ago

I marked this as stale due to inactivity. → More info

rhendric commented 1 year ago

Is #234201 merging enough to close this issue?

jtojnar commented 1 year ago

The issue will persist until the packages switch to the new convention.

jtojnar commented 1 year ago

And probably also support code like https://github.com/NixOS/nixpkgs/blob/3a1e7765172c444c348dde552ab4fa549ef59b27/pkgs/common-updater/unstable-updater.nix.