search

NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
17.34k stars 13.58k forks source link

python3Packages: get rid of uses of nose #326513

Open emilazy opened 2 months ago

emilazy commented 2 months ago

nose is abandoned upstream and has caused us untold headaches recently (admittedly mostly unrelated to the code itself). We’ve stopped the bleeding by getting https://github.com/NixOS/nixpkgs/pull/325968 into staging thanks to the incredible work of @jchv, but it’s still broken on master and we ought to do some tidying up so we’re not dragging this around forever and perhaps even to unblock some stuff on master while the staging cycle progresses. Many packages don’t actually need nose any more, despite us including it in our dependency lists. Active upstreams should have already moved to pytest or something else. Some packages will have patches available; if we’re really feeling generous we could send patches to upstream ourselves. Others may be completely unmaintained packages that we could think about dropping.

Please make sure to migrate packages to pyproject = true; when working on them. Additionally, if the upstream is unmaintained, there are no significant reverse dependencies in‐tree or on Wheelodex, and it requires patching or there is no particular reason to imagine we’d want to keep it, consider just dropping the package. For the same reasons we want to get rid of nose, we should take this as an opportunity to reduce future maintenance burden and drop broken, unused, or long‐obsolete packages.

@dotlambda has already done some great work on this – see https://github.com/NixOS/nixpkgs/pull/325847, https://github.com/NixOS/nixpkgs/pull/325872, https://github.com/NixOS/nixpkgs/pull/325885, and https://github.com/NixOS/nixpkgs/pull/326164 – but I thought it’d be good to have a master list of all affected packages so we can tackle it collectively. If you open a PR for one of these, please edit this issue to link it beside the relevant package(s).

This was generated from staging commit 83881cb6e9b660cbca5241380c031ab2b69d6e1e with the following awful fish one‐liner: for pkg in (rg -l '\bnose3?\b' | rg -v '/(nose[23]?|cufflinks|enocean|jaconv|annoy|hdmedians|flaky|pycron|snapshottest|ssh-mitm|colour|influxdb)/|(python|cran)-(packages|aliases)\.nix' | sed 's|.*/\([^/]*\)/\([^/]*\)$|\1|'); rg "/$pkg"'[ \n{]' pkgs/top-level/python-packages.nix | awk '{print "python3Packages." $1}'; test $pipestatus[1] = 0 || rg "/$pkg"'[ \n{]' pkgs/top-level/all-packages.nix | awk '{print $1}'; test $pipestatus[1] = 0 || echo "$pkg"; end | sort -u | sed 's/^/* [ ] `/; s/$/`/'; feel free to edit to update the list.

Sigmanificient commented 1 month ago

@emilazy in your list, candidates1 seems like a script error, i cant find it in nixpkgs

emilazy commented 1 month ago

Uh, yeah I have no idea where that came from.

aws-google-auth may be dead: cevoaustralia/aws-google-auth#284

Probably just drop it, although we’re not running its tests anyway.

Sigmanificient commented 4 weeks ago

2 packages that have been already done @emilazy

mmorley0395 commented 4 weeks ago

new to nix with much to learn- but quick question. will these be backported into 24.05 when complete? or better to point a flake to master or unstable for packages that are fixed? thanks!

emilazy commented 3 weeks ago

24.05 uses Python 3.11 by default, so the packages here should mostly not be an issue. If you’re using Python 3.12, I would recommend using the unstable release until 24.11 is released.

(Also, most of these packages do already work with 3.12 on 24.11 as we have patched nose ourselves; we’re just trying to get them cleaned up.)

mmorley0395 commented 3 weeks ago

Interesting-- I was using nix-shell and for some reason my nix-channel was set to unstable so it was pulling from there and therefore using 3.12 i guess. I don't need 3.12 specifically, so updated the channel and all is well now. Thanks for taking the time to explain!

Sigmanificient commented 3 weeks ago

Up to date list

131 items: 82.44% done, 10.69% in progress, 11 waiting

python2Packages

python3Packages

Sigmanificient commented 3 weeks ago

umcompyle6 has doCheck = pythonOlder "3.9"; but when using nix-build -A python312.umcompyle6 tests are still being run, am I missing something @emilazy?

Sigmanificient commented 3 weeks ago

@emilazy should we drop more-itertools?

SuperSandro2000 commented 2 weeks ago

more-itertools

We can't, it is to widely used. Or do you mean the python2Package? Not sure what is even using that at this point.

Sigmanificient commented 2 weeks ago

more-itertools

We can't, it is to widely used. Or do you mean the python2Package? Not sure what is even using that at this point.

yup the python2Package

SuperSandro2000 commented 2 weeks ago

Try removing it and see what breaks. I would be in support to drop eg https://github.com/NixOS/nixpkgs/blob/21576aba43e604a95cc45c25248f39f3989ebe36/pkgs/servers/neard/default.nix

but we need to keep gimp intact https://github.com/NixOS/nixpkgs/blob/21576aba43e604a95cc45c25248f39f3989ebe36/pkgs/applications/graphics/gimp/default.nix#L137

I didn't find anything else uses python2 packages on the fast but I didn't search for all variants.

emilazy commented 2 weeks ago

Ignore Python 2; it is even more obsolete than nose. GIMP will be solved soon and I have plans for the rest of Python 2.