search

NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
17.32k stars 13.56k forks source link

Migrate python packages from nose to pynose #311054

Closed mweinelt closed 2 months ago

mweinelt commented 4 months ago

The nose test runner will never run on Python 3.12, and we're replacing it with pynose instead. On top of that, packages relying on nose have largely been unmaintained and might be dead leafs or using another test framework entirely. It is also to be expected that they need to be converted to the PEP517 builder in the process of fixing them.

The following packages are in the python3Packages scope and should be addressed first. The final goal is the removal of nose and related packages.

MostAwesomeDude commented 2 months ago

In mdmintz/pynose#16 and mdmintz/pynose#26 it was revealed that pynose is not licenses.mit as announced. I think it should probably inherit the license from nose, which is currently licenses.lgpl3. If you'd like me to fix this, ping me and I'll send a PR referencing this issue.

eli-schwartz commented 2 months ago

Given that pynose is illegal, copyright-violating software, which by virtue of violating the GPL has lost its right to use the software under the terms of the GPL, it's not remotely clear to me that NixOS is legally allowed to redistribute the tainted edition of the software at all.

I guess I'm not really surprised that some people automatically assume they can do so regardless.

mweinelt commented 2 months ago

I'm working on a treewide removal of pynose.

SomeoneSerge commented 2 months ago

In https://github.com/mdmintz/pynose/issues/16 and https://github.com/mdmintz/pynose/issues/26 it was revealed that pynose is not licenses.mit as announced.

I'm not familiar with nose/pynose, but I'm wondering if something like lib.licenses.unfree // { shortName = "disputed"; } could be adopted for similar situations. Ofc this would nonetheless require making the package a leaf node, possibly rendering the exercise futile

mweinelt commented 2 months ago

possibly rendering the exercise futile

That.

eli-schwartz commented 2 months ago

Nonfree typically means the upstream project legally has the right to be the upstream project, they simply choose to only grant you the ability to use that code under a non OSI approved license.

Where does the concept of a "disputed" license value come from? Under what terms and conditions could or would a user install such a thing, even as a leaf package?

SomeoneSerge commented 2 months ago

Nonfree typically means the upstream project legally has the right to be the upstream project

That sounds reasonable in the context of natural languages. In the context of the metadata that comes with derivations, it's just a single boolean which an individual maintainer could set for whatever reasons... the minimal semantics I could infer from it is "not to be redistributed by Hydra" and "just because you can build it doesn't mean anyone's allowed you to use or distribute it"

Under what terms and conditions could or would a user install such a thing, even as a leaf package?

On terms of taking their own risk; I wouldn't mind shortName = "illegal" either

mweinelt commented 2 months ago

The upstream has no right to redistribute the code under the given license. That is the issue we run into.

eli-schwartz commented 2 months ago

That sounds reasonable in the context of natural languages. In the context of the metadata that comes with derivations, it's just a single boolean which an individual maintainer could set for whatever reasons...

In the context of naturally attempting to convey useful and usable semantic meaning for conventional language concepts, it sounds reasonable to say "if an individual maintainer sets a deceptive value that deceives the user into associating the meaning with that of projects which are under proprietary, but entirely legal, licenses -- the individual maintainer is acting against the best interests of the project and shall receive an infraction by whichever disciplinary measure the distribution in question uses to penalize bad behavior".

I don't quite see the purpose in defending the notion of packaging illegal software using language intended to seem similar to commercial+proprietary redistributables. Is this a thought experiment or a practical desire?

SomeoneSerge commented 2 months ago

Sorry @eli-schwartz, I hadn't read the entirety of 16 and 26 when I first commented, and now, maybe as a result, we're mixing a discussion about implementation details and one about policies.

Is this a thought experiment or a practical desire?

Semi-practical. We've several expressions that describe how to build certain pieces of "legal" software for which, however, we do not have (but the user might) a legal way of obtaining the sources or for which we wouldn't have the right to distribute the results. This is not quite the same as the present situation, but bear with me. These expressions would reference a generally unavailable source, but (a) we wouldn't build them in the CI (ergo never access the source nor distribute), (b) we'd mark them so that they don't evaluate before the user explicitly opts in.

On the implementation side, and I don't particularly like that we confound the names, the latter is practically done by resetting free to false in the "license" description.

nix-repl> runCommand "something" { meta.license = [ { free = false; } ]; } "touch $out"
...
       error: Package ‘something’ in /nix/store/4p0avw1s3vf27hspgqsrqs37gxk4i83i-source/pkgs/build-support/trivial-builders/default.nix:68 has an unfree license (‘unknown’), refusing to evaluate.
...
nix-repl> runCommand "something" { meta.license = [ (lib.licenses.unfree // { shortName = "illegal"; }) ]; } "touch $out"
...
       error: Package ‘something’ in /nix/store/4p0avw1s3vf27hspgqsrqs37gxk4i83i-source/pkgs/build-support/trivial-builders/default.nix:68 has an unfree license (‘illegal’), refusing to evaluate.
...
       Alternatively you can configure a predicate to allow specific packages:
         { nixpkgs.config.allowUnfreePredicate = pkg: builtins.elem (lib.getName pkg) [
             "something"
...

While this error message would be wrong of course (maybe now's a good time to reword it), since there's no, as such, a license granting anything to anyone, the way I read it it does attempt to communicate that "whatever the licensing situation is, you're not automatically permitted to do anything with this software and it's your task to find out the terms and get the consents". Essentially, when I set the license value, I treat free = true as bearing legal meaning (the authors give explicit consent to do FOSS things; in particular, it's sufficient to automatically assume that one may at least build the derivation, but otherwise still has to consult license.meta.url), whereas free = false to me is broader and only means the absence of something.

The licensing information in the pynose expression we have right now is simply wrong. At the very least we'd have to delete mit from the list (resulting in "refusing to evaluate" and "license ('unknown')"), but then we can't depend on it in other packages (lest the CI refuse to build them too). Once these reverse-dependencies are rewritten not to depend on pynose, there'll not be much value in keeping pynose around...

The policy question of when it's worthwhile or acceptable to provide expressions for software which we can't reference a "legally available" source for (proprietary, unlicensed, ...), and whether in particular it's acceptable to reference sources we do know to violate other authors' dissent is global. While this might be worth answering globally "once and for all", we already have, by the previous paragraph, enough local information to remove the package...

EDIT: I seem to have deviated from the original point here. Back to "a thought experiment or a practical desire". Representing correct knowledge about upstream projects definitely is a goal. The present issue is an edge case where our tooling (meta.license) can't meaningfully represent our knowledge, and it's a reminder that our tooling hasn't been entirely adequate for modeling points in the interior of the spectrum (like the nvidia stuff), and it wouldn't have been adequate for representing e.g. unlicensed projects either. So, "semi-practical"

eli-schwartz commented 2 months ago

Interesting implementation details!

In gentoo, there is a global repository directory containing license terms and conditions, and the file profiles/license_groups maps many registered license to some meta-group such as (random sampling):

Users would have to specify which licenses they accept to use (the default is ACCEPT_LICENSE="-* @FREE"). Most people probably do not add ACCEPT_LICENSE="JetBrains-business" or some such, so the result is if you try to install certain packages...

!!! All ebuilds that could satisfy "goland" have been masked.
!!! One of the following masked packages is required to complete your request:
- dev-util/goland-2022.3.3-r1::gentoo (masked by: || ( JetBrains-business JetBrains-classroom JetBrains-educational JetBrains-individual ) license(s), ~amd64 keyword)
A copy of the 'JetBrains-business' license is located at '/var/db/repos/gentoo/licenses/JetBrains-business'.

A copy of the 'JetBrains-classroom' license is located at '/var/db/repos/gentoo/licenses/JetBrains-classroom'.

A copy of the 'JetBrains-educational' license is located at '/var/db/repos/gentoo/licenses/JetBrains-educational'.

A copy of the 'JetBrains-individual' license is located at '/var/db/repos/gentoo/licenses/JetBrains-individual'.

For more information, see the MASKED PACKAGES section in the emerge
man page or refer to the Gentoo Handbook.

Project policy is to "ask the opinion of others before committing a new license, if you're not sure about the license". This is somewhat flexible obviously. The addition of new licenses is, incidentally, easy to coordinate and watch for changes. :D Getting the license wrong is a very human mistake, adding something that isn't a license at all would probably raise a few eyebrows.

There's no real room to mark a package as nonfree without stating the specific reason it isn't free, since "unknown" licenses are banned by CI-enforced policy.

The license groups in particular are very neat. It encourages accurate specification of "terms and conditions" independent of "is this license one that I consider free", since you can have arbitrarily grouped or recursively-grouped license sets to describe policy with, instead.

eli-schwartz commented 2 months ago

Ah, w.r.t. the edit: one could ACCEPT_LICENSE="NVIDIA-CUDA", or since that is part of the @EULA group, ACCEPT_LICENSE="@EULA" if one doesn't mind an EULA in general.

SomeoneSerge commented 2 months ago

Nixpkgs is orderly enough about "free and opensource" licenses (there's a list maintained in a single place which other expressions reference, e.g. license = [ lib.licenses.mit ]), but the rest can be fairly ad hoc at times. We'll surely look at Gentoo for inspiration when we try to bring some order to this part of Nixpkgs, or when we discuss the policy

As for NVidia, one approach usable today could look something like this: https://github.com/NixOS/nixpkgs/blob/7a95a8948b9ae171337bbf2794459dbe167032ed/pkgs/top-level/release-cuda.nix#L17-L28

emilazy commented 1 month ago

Spiritual successor: https://github.com/NixOS/nixpkgs/issues/326513

mdmintz commented 1 month ago

This should hopefully help clear up any confusion: https://github.com/mdmintz/pynose/issues/36