search

DavHau / mach-nix

Create highly reproducible python environments
MIT License
868 stars 106 forks source link

Wrong requirements conflict when using mkPythonShell #418

Open yajo opened 2 years ago

yajo commented 2 years ago

Create this default.nix file:

let
  mach-nix = import (builtins.fetchGit {
    url = "https://github.com/DavHau/mach-nix";
    rev = "7b0e7f6f58f7332c1b8347e2673036eada1a6998";
  }) {};
  src = builtins.fetchGit {
    url = "https://github.com/odoo/odoo";
    rev = "a08f4e00715dba4a0447dce879b46da0e581669e";
    ref = "15.0";
  };
in mach-nix.buildPythonPackage {
  inherit src;
  python = "python38";
  requirements = builtins.readFile "${src}/requirements.txt";
  requirementsExtra = "mock";
}

Run it:

> nix-shell --command "which python"
trace: 
 automatically detected pname: 'odoo'
trace: 
 automatically detected version: '15.0'
trace: 
applying fix 'no-rust-build' (nativeBuildInputs) for cryptography:3.3.2

trace: removing dependency python3.8-setuptools-rust-0.12.1 from cryptography
trace: removing dependency python3.8-packaging-21.3 from cryptography
trace: removing dependency python3.8-setuptools-scm-6.3.2 from pyusb
Sourcing python-remove-tests-dir-hook
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-bin-bytecode-hook.sh
Sourcing setuptools-build-hook
Using setuptoolsBuildPhase
Using setuptoolsShellHook
Sourcing pip-install-hook
Using pipInstallPhase
Sourcing python-imports-check-hook.sh
Using pythonImportsCheckPhase
Sourcing python-namespaces-hook
Executing setuptoolsShellHook
Finished executing setuptoolsShellHook
/nix/store/c502m2h2yfqymf9bg4jvvhj038pfimcb-python3-3.8.12/bin/python

Build it:

> nix-build
trace: 
 automatically detected pname: 'odoo'
trace: 
 automatically detected version: '15.0'
trace: 
applying fix 'no-rust-build' (nativeBuildInputs) for cryptography:3.3.2

trace: removing dependency python3.8-setuptools-rust-0.12.1 from cryptography
trace: removing dependency python3.8-packaging-21.3 from cryptography
trace: removing dependency python3.8-setuptools-scm-6.3.2 from pyusb
/nix/store/r62rl5r911vc017x6jljh2a47d5c350k-python3.8-odoo-15.0

Cool!

Now let's say I want a shell with that and other tools. Add this shell.nix:

let
  mach-nix = import (builtins.fetchGit {
    url = "https://github.com/DavHau/mach-nix";
    rev = "7b0e7f6f58f7332c1b8347e2673036eada1a6998";
  }) {};
  odoo = (import ./.);
in mach-nix.mkPythonShell {
  python = "python38";
  packagesExtra = [odoo];
}

Run it:

> nix-shell --command "which python"
trace: 
 automatically detected pname: 'odoo'
trace: 
 automatically detected version: '15.0'
building '/nix/store/pfzjq2gbcrp516d4a3mz2yjxbli7p8mi-mach_nix_file.drv'...

Some requirements could not be resolved.
Top level requirements: 
  odoo
Providers:
  {'_default': ['wheel', 'sdist', 'nixpkgs'],
   'gdal': ['nixpkgs'],
   'odoo': ['nixpkgs'],
   'pip': ['nixpkgs', 'sdist'],
   'setupmeta': ['wheel'],
   'setuptools': ['nixpkgs'],
   'wheel': ['nixpkgs', 'sdist']}
Mach-nix version: master
Python: 3.8.12
Cause: None
The requirements which caused the error:
  freezegun (<SpecifierSet('==0.3.11')>,) python_version < '3.8' - parent: odoo:15.0
  freezegun (<SpecifierSet('==0.3.15')>,) python_version >= '3.8' - parent: odoo:15.0

The given requirements might contain package versions which are not yet part of the dependency DB
currently used. The DB can be updated by specifying 'pypiDataRev' when importing mach-nix.
For examples see: https://github.com/DavHau/mach-nix/blob/master/examples.md

error: builder for '/nix/store/pfzjq2gbcrp516d4a3mz2yjxbli7p8mi-mach_nix_file.drv' failed with exit code 1
(use '--show-trace' to show detailed location information)
```sh > nix-shell --command "which python" --show-trace trace: automatically detected pname: 'odoo' trace: automatically detected version: '15.0' building '/nix/store/pfzjq2gbcrp516d4a3mz2yjxbli7p8mi-mach_nix_file.drv'... Some requirements could not be resolved. Top level requirements: odoo Providers: {'_default': ['wheel', 'sdist', 'nixpkgs'], 'gdal': ['nixpkgs'], 'odoo': ['nixpkgs'], 'pip': ['nixpkgs', 'sdist'], 'setupmeta': ['wheel'], 'setuptools': ['nixpkgs'], 'wheel': ['nixpkgs', 'sdist']} Mach-nix version: master Python: 3.8.12 Cause: None The requirements which caused the error: freezegun (,) python_version < '3.8' - parent: odoo:15.0 freezegun (,) python_version >= '3.8' - parent: odoo:15.0 The given requirements might contain package versions which are not yet part of the dependency DB currently used. The DB can be updated by specifying 'pypiDataRev' when importing mach-nix. For examples see: https://github.com/DavHau/mach-nix/blob/master/examples.md error: builder for '/nix/store/pfzjq2gbcrp516d4a3mz2yjxbli7p8mi-mach_nix_file.drv' failed with exit code 1 … while realising the context of path '/nix/store/a0briqv8av1sava20dpxqvcixgab8rq8-mach_nix_file/share/mach_nix_file.nix' at /nix/store/4vfx4nqr1nsqibyxcavxvkb5wsqxx9x6-source/mach_nix/nix/lib.nix:180:16: 179| file = "${compileExpression args}/share/mach_nix_file.nix"; 180| result = import file { inherit (args) pkgs python; }; | ^ 181| manylinux = … while evaluating the attribute 'overrides' at /nix/store/4vfx4nqr1nsqibyxcavxvkb5wsqxx9x6-source/mach_nix/nix/lib.nix:187:7: 186| in { 187| overrides = result.overrides manylinux autoPatchelfHook; | ^ 188| select_pkgs = result.select_pkgs; … while evaluating 'composeExtensions' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:75:18: 74| composeExtensions = 75| f: g: final: prev: | ^ 76| let fApplied = f final prev; … from call site at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:76:22: 75| f: g: final: prev: 76| let fApplied = f final prev; | ^ 77| prev' = prev // fApplied; … while evaluating 'composeExtensions' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:75:18: 74| composeExtensions = 75| f: g: final: prev: | ^ 76| let fApplied = f final prev; … from call site at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:76:22: 75| f: g: final: prev: 76| let fApplied = f final prev; | ^ 77| prev' = prev // fApplied; … while evaluating 'composeExtensions' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:75:18: 74| composeExtensions = 75| f: g: final: prev: | ^ 76| let fApplied = f final prev; … from call site at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:76:22: 75| f: g: final: prev: 76| let fApplied = f final prev; | ^ 77| prev' = prev // fApplied; … while evaluating 'composeExtensions' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:75:18: 74| composeExtensions = 75| f: g: final: prev: | ^ 76| let fApplied = f final prev; … from call site at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:76:22: 75| f: g: final: prev: 76| let fApplied = f final prev; | ^ 77| prev' = prev // fApplied; … while evaluating 'composeExtensions' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:75:18: 74| composeExtensions = 75| f: g: final: prev: | ^ 76| let fApplied = f final prev; … from call site at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:76:22: 75| f: g: final: prev: 76| let fApplied = f final prev; | ^ 77| prev' = prev // fApplied; … while evaluating 'composeExtensions' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:75:18: 74| composeExtensions = 75| f: g: final: prev: | ^ 76| let fApplied = f final prev; … from call site at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:76:22: 75| f: g: final: prev: 76| let fApplied = f final prev; | ^ 77| prev' = prev // fApplied; … while evaluating 'composeExtensions' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:75:18: 74| composeExtensions = 75| f: g: final: prev: | ^ 76| let fApplied = f final prev; … from call site at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:76:22: 75| f: g: final: prev: 76| let fApplied = f final prev; | ^ 77| prev' = prev // fApplied; … while evaluating 'composeExtensions' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:75:18: 74| composeExtensions = 75| f: g: final: prev: | ^ 76| let fApplied = f final prev; … from call site at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:76:22: 75| f: g: final: prev: 76| let fApplied = f final prev; | ^ 77| prev' = prev // fApplied; … while evaluating 'composeExtensions' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:75:18: 74| composeExtensions = 75| f: g: final: prev: | ^ 76| let fApplied = f final prev; … from call site at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:76:22: 75| f: g: final: prev: 76| let fApplied = f final prev; | ^ 77| prev' = prev // fApplied; … while evaluating 'composeExtensions' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:75:18: 74| composeExtensions = 75| f: g: final: prev: | ^ 76| let fApplied = f final prev; … from call site at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:76:22: 75| f: g: final: prev: 76| let fApplied = f final prev; | ^ 77| prev' = prev // fApplied; … while evaluating 'composeExtensions' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:75:18: 74| composeExtensions = 75| f: g: final: prev: | ^ 76| let fApplied = f final prev; … from call site at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:76:22: 75| f: g: final: prev: 76| let fApplied = f final prev; | ^ 77| prev' = prev // fApplied; … while evaluating 'composeExtensions' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:75:18: 74| composeExtensions = 75| f: g: final: prev: | ^ 76| let fApplied = f final prev; … from call site at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:76:22: 75| f: g: final: prev: 76| let fApplied = f final prev; | ^ 77| prev' = prev // fApplied; … while evaluating 'composeExtensions' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:75:18: 74| composeExtensions = 75| f: g: final: prev: | ^ 76| let fApplied = f final prev; … from call site at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:76:22: 75| f: g: final: prev: 76| let fApplied = f final prev; | ^ 77| prev' = prev // fApplied; … while evaluating 'composeExtensions' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:75:18: 74| composeExtensions = 75| f: g: final: prev: | ^ 76| let fApplied = f final prev; … from call site at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:76:22: 75| f: g: final: prev: 76| let fApplied = f final prev; | ^ 77| prev' = prev // fApplied; … while evaluating 'composeExtensions' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:75:18: 74| composeExtensions = 75| f: g: final: prev: | ^ 76| let fApplied = f final prev; … from call site at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:76:22: 75| f: g: final: prev: 76| let fApplied = f final prev; | ^ 77| prev' = prev // fApplied; … while evaluating 'composeExtensions' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:75:18: 74| composeExtensions = 75| f: g: final: prev: | ^ 76| let fApplied = f final prev; … from call site at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:78:22: 77| prev' = prev // fApplied; 78| in fApplied // g final prev'; | ^ 79| … while evaluating 'composeExtensions' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:75:18: 74| composeExtensions = 75| f: g: final: prev: | ^ 76| let fApplied = f final prev; … from call site at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:69:67: 68| # 69| extends = f: rattrs: self: let super = rattrs self; in super // f self super; | ^ 70| … while evaluating 'extends' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/fixed-points.nix:69:24: 68| # 69| extends = f: rattrs: self: let super = rattrs self; in super // f self super; | ^ 70| … from call site at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/customisation.nix:233:14: 232| spliced = extra spliced0 // spliced0 // keep self; 233| self = f self // { | ^ 234| newScope = scope: newScope (spliced // scope); … while evaluating 'makeScopeWithSplicing' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/customisation.nix:222:80: 221| hopefully it helps a little bit. */ 222| makeScopeWithSplicing = splicePackages: newScope: otherSplices: keep: extra: f: | ^ 223| let … from call site at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/pkgs/development/interpreters/python/default.nix:79:12: 78| aliases = self: super: lib.optionalAttrs (config.allowAliases or true) (import ../../../top-level/python-aliases.nix lib self super); 79| in lib.makeScopeWithSplicing | ^ 80| pkgs.splicePackages … while evaluating anonymous lambda at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/pkgs/development/interpreters/python/default.nix:25:10: 24| pythonPackages = callPackage 25| ({ pkgs, stdenv, python, overrides }: let | ^ 26| pythonPackagesFun = import ../../../top-level/python-packages.nix { … from call site at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/customisation.nix:69:16: 68| let 69| result = f origArgs; | ^ 70| … while evaluating 'makeOverridable' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/customisation.nix:67:24: 66| */ 67| makeOverridable = f: origArgs: | ^ 68| let … from call site at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/customisation.nix:121:8: 120| auto = builtins.intersectAttrs (lib.functionArgs f) autoArgs; 121| in makeOverridable f (auto // args); | ^ 122| … while evaluating 'callPackageWith' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/lib/customisation.nix:117:35: 116| */ 117| callPackageWith = autoArgs: fn: args: | ^ 118| let … from call site at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/pkgs/development/interpreters/python/default.nix:24:24: 23| }: let 24| pythonPackages = callPackage | ^ 25| ({ pkgs, stdenv, python, overrides }: let … while evaluating the attribute 'passAsFile' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/pkgs/build-support/buildenv/default.nix:77:5: 76| # XXX: The size is somewhat arbitrary 77| passAsFile = if builtins.stringLength pkgs >= 128*1024 then [ "pkgs" ] else [ ]; | ^ 78| } … while evaluating the attribute 'passAsFile' of the derivation 'python3-3.8.12-env' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/pkgs/stdenv/generic/make-derivation.nix:205:7: 204| // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) { 205| name = | ^ 206| let … while evaluating the attribute 'nativeBuildInputs' of the derivation 'interactive-python3-3.8.12-environment' at /nix/store/5n402azp0s9vza4rziv4z5y88v2cv1mq-nixpkgs/pkgs/stdenv/generic/make-derivation.nix:205:7: 204| // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) { 205| name = | ^ 206| let ```

That error doesn't make much sense, since the requirements seem perfectly valid and resolvable, as they were for the package.

yajo commented 2 years ago

I found the workaround. Use this shell.nix:

let
  mach-nix = import (builtins.fetchGit {
    url = "https://github.com/DavHau/mach-nix";
    rev = "7b0e7f6f58f7332c1b8347e2673036eada1a6998";
  }) {};
  odoo = (import ./.);
in (odoo.pythonModule.withPackages (ps: [odoo])).env

It works as expected:

> nix-shell
trace: removing dependency python3.8-packaging-21.3 from cryptography
trace: 
applying fix 'no-rust-build' (nativeBuildInputs) for cryptography:3.3.2

trace: removing dependency python3.8-setuptools-rust-0.12.1 from cryptography
trace: removing dependency python3.8-setuptools-scm-6.3.2 from pyusb
trace: 
 automatically detected pname: 'odoo'
trace: 
 automatically detected version: '15.0'

[nix-shell:~/prodevel/nixdoo/test]$ odoo --version
Odoo Server 15.0

[nix-shell:~/prodevel/nixdoo/test]$ python -c 'import odoo; print(odoo.release.version)'
15.0

[nix-shell:~/prodevel/nixdoo/test]$ 
exit