search

NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
18.06k stars 14.1k forks source link

Build failure: python3Packages.tpm2-pytss #252023

Closed ezracelli closed 1 year ago

ezracelli commented 1 year ago

Steps To Reproduce

[user@system:~]$ nix build github:nixos/nixpkgs/a999c1cc0c9eb2095729d5aa03e0d8f7ed256780#python3Packages.tpm2-pytss # current tip of nixos-unstable

Build log

nix log /nix/store/xglq4zgrfjp92jrjn1d740wgssrzbqpy-python3.10-tpm2-pytss-2.1.0.drv ``` Sourcing python-remove-tests-dir-hook Sourcing python-catch-conflicts-hook.sh Sourcing python-remove-bin-bytecode-hook.sh Sourcing setuptools-build-hook Using setuptoolsBuildPhase Using setuptoolsShellHook Sourcing pypa-install-hook Using pypaInstallPhase Sourcing python-imports-check-hook.sh Using pythonImportsCheckPhase Sourcing python-namespaces-hook Sourcing python-catch-conflicts-hook.sh Sourcing setuptools-check-hook Using setuptoolsCheckPhase Sourcing pytest-check-hook Using pytestCheckPhase Removing setuptoolsCheckPhase @nix { "action": "setPhase", "phase": "unpackPhase" } unpacking sources unpacking source archive /nix/store/1d8j2gc9p61945mv0ig07dmsggxl7s7j-tpm2-pytss-2.1.0.tar.gz source root is tpm2-pytss-2.1.0 setting SOURCE_DATE_EPOCH to timestamp 1673278912 of file tpm2-pytss-2.1.0/src/tpm2_pytss.egg-info/top_level.txt @nix { "action": "setPhase", "phase": "patchPhase" } patching sources applying patch /nix/store/zjz9yv9f34zx0imrsgqhmyib23hkk1d6-099c069f28cfcd0a3019adebfeafa976f9395221.patch patching file setup.py applying patch /nix/store/ddlvkwj9vazi8s9pn59gjy5q1nxlk6fq-97289a08ddf44f7bdccdd122d6055c69e12dc584.patch patching file src/tpm2_pytss/TCTILdr.py patching file test/TSS2_BaseTest.py patching file test/test_tcti.py applying patch /nix/store/9hl05vs00afl1kb2q2h8xgqxq8x33dhv-test-new-cryptography.patch patching file test/test_crypto.py applying patch /nix/store/6gnm6ndfh4wjbw5g3q36ca5j2d0ri7dr-fapi-config.patch patching file src/tpm2_pytss/FAPI.py @nix { "action": "setPhase", "phase": "updateAutotoolsGnuConfigScriptsPhase" } updateAutotoolsGnuConfigScriptsPhase @nix { "action": "setPhase", "phase": "configurePhase" } configuring no configure script, doing nothing @nix { "action": "setPhase", "phase": "buildPhase" } building Executing setuptoolsBuildPhase /nix/store/wyj4pwzaa11rlm3dkw4ywpl6l747g5sj-python3.10-setuptools-68.0.0/lib/python3.10/site-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated. !! ******************************************************************************** Requirements should be satisfied by a PEP 517 installer. If you are using pip, you can try `pip install --use-pep517`. ******************************************************************************** !! dist.fetch_build_eggs(dist.setup_requires) adding path: /build/tpm2-pytss-2.1.0/scripts running bdist_wheel running build running build_py creating build creating build/lib.linux-x86_64-cpython-310 creating build/lib.linux-x86_64-cpython-310/tpm2_pytss copying src/tpm2_pytss/ESAPI.py -> build/lib.linux-x86_64-cpython-310/tpm2_pytss copying src/tpm2_pytss/constants.py -> build/lib.linux-x86_64-cpython-310/tpm2_pytss copying src/tpm2_pytss/TCTISPIHelper.py -> build/lib.linux-x86_64-cpython-310/tpm2_pytss copying src/tpm2_pytss/types.py -> build/lib.linux-x86_64-cpython-310/tpm2_pytss copying src/tpm2_pytss/FAPI.py -> build/lib.linux-x86_64-cpython-310/tpm2_pytss copying src/tpm2_pytss/TSS2_Exception.py -> build/lib.linux-x86_64-cpython-310/tpm2_pytss copying src/tpm2_pytss/utils.py -> build/lib.linux-x86_64-cpython-310/tpm2_pytss copying src/tpm2_pytss/__init__.py -> build/lib.linux-x86_64-cpython-310/tpm2_pytss copying src/tpm2_pytss/fapi_info.py -> build/lib.linux-x86_64-cpython-310/tpm2_pytss copying src/tpm2_pytss/tsskey.py -> build/lib.linux-x86_64-cpython-310/tpm2_pytss copying src/tpm2_pytss/encoding.py -> build/lib.linux-x86_64-cpython-310/tpm2_pytss copying src/tpm2_pytss/TCTILdr.py -> build/lib.linux-x86_64-cpython-310/tpm2_pytss copying src/tpm2_pytss/policy.py -> build/lib.linux-x86_64-cpython-310/tpm2_pytss copying src/tpm2_pytss/TCTI.py -> build/lib.linux-x86_64-cpython-310/tpm2_pytss creating build/lib.linux-x86_64-cpython-310/tpm2_pytss/internal copying src/tpm2_pytss/internal/templates.py -> build/lib.linux-x86_64-cpython-310/tpm2_pytss/internal copying src/tpm2_pytss/internal/utils.py -> build/lib.linux-x86_64-cpython-310/tpm2_pytss/internal copying src/tpm2_pytss/internal/__init__.py -> build/lib.linux-x86_64-cpython-310/tpm2_pytss/internal copying src/tpm2_pytss/internal/crypto.py -> build/lib.linux-x86_64-cpython-310/tpm2_pytss/internal running build_ext generating cffi module 'build/temp.linux-x86_64-cpython-310/tpm2_pytss._libtpm2_pytss.c' creating build/temp.linux-x86_64-cpython-310 building 'tpm2_pytss._libtpm2_pytss' extension creating build/temp.linux-x86_64-cpython-310/build creating build/temp.linux-x86_64-cpython-310/build/temp.linux-x86_64-cpython-310 gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -I/nix/store/dn5s2bxwvrwn0pjbq3kyqw4lkfbd4az8-libxcrypt-4.4.36/include -fPIC -I/nix/store/bh6yvqz3c4whg2kvb3akdnyv7mn5if22-tpm2-tss-4.0.1-dev/include -I/nix/store/bh6yvqz3c4whg2kvb3akdnyv7mn5if22-tpm2-tss-4.0.1-dev/include/tss2 -I/nix/store/bh6yvqz3c4whg2kvb3akdnyv7mn5if22-tpm2-tss-4.0.1-dev/include -I/nix/store/bh6yvqz3c4whg2kvb3akdnyv7mn5if22-tpm2-tss-4.0.1-dev/include/tss2 -I/nix/store/bh6yvqz3c4whg2kvb3akdnyv7mn5if22-tpm2-tss-4.0.1-dev/include -I/nix/store/bh6yvqz3c4whg2kvb3akdnyv7mn5if22-tpm2-tss-4.0.1-dev/include/tss2 -I/nix/store/bh6yvqz3c4whg2kvb3akdnyv7mn5if22-tpm2-tss-4.0.1-dev/include -I/nix/store/bh6yvqz3c4whg2kvb3akdnyv7mn5if22-tpm2-tss-4.0.1-dev/include/tss2 -I/nix/store/bh6yvqz3c4whg2kvb3akdnyv7mn5if22-tpm2-tss-4.0.1-dev/include -I/nix/store/bh6yvqz3c4whg2kvb3akdnyv7mn5if22-tpm2-tss-4.0.1-dev/include/tss2 -I/nix/store/bh6yvqz3c4whg2kvb3akdnyv7mn5if22-tpm2-tss-4.0.1-dev/include -I/nix/store/bh6yvqz3c4whg2kvb3akdnyv7mn5if22-tpm2-tss-4.0.1-dev/include/tss2 -I/nix/store/bh6yvqz3c4whg2kvb3akdnyv7mn5if22-tpm2-tss-4.0.1-dev/include -I/nix/store/bh6yvqz3c4whg2kvb3akdnyv7mn5if22-tpm2-tss-4.0.1-dev/include/tss -I/nix/store/pkj7cgmz66assy7l18zc7j992npb41nx-python3-3.10.12/include/python3.10 -c build/temp.linux-x86_64-cpython-310/tpm2_pytss._libtpm2_pytss.c -o build/temp.linux-x86_64-cpython-310/build/temp.linux-x86_64-cpython-310/tpm2_pytss._libtpm2_pytss.o gcc -shared -L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-zlib-1.2.13/lib -L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-bzip2-1.0.8/lib -L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-expat-2.5.0/lib -L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-xz-5.4.4/lib -L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-libffi-3.4.4/lib -L/nix/store/dn5s2bxwvrwn0pjbq3kyqw4lkfbd4az8-libxcrypt-4.4.36/lib -L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-gdbm-1.23/lib -L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-sqlite-3.42.0/lib -L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-readline-8.2p1/lib -L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-ncurses-6.4/lib -L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-openssl-3.0.10/lib -L/nix/store/dbr96xib9c1d3yqi5iqccz7a89r3qw18-tzdata-2023c/lib -L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-zlib-1.2.13/lib -L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-bzip2-1.0.8/lib -L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-expat-2.5.0/lib -L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-xz-5.4.4/lib -L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-libffi-3.4.4/lib -L/nix/store/dn5s2bxwvrwn0pjbq3kyqw4lkfbd4az8-libxcrypt-4.4.36/lib -L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-gdbm-1.23/lib -L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-sqlite-3.42.0/lib -L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-readline-8.2p1/lib -L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-ncurses-6.4/lib -L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-openssl-3.0.10/lib -L/nix/store/dbr96xib9c1d3yqi5iqccz7a89r3qw18-tzdata-2023c/lib build/temp.linux-x86_64-cpython-310/build/temp.linux-x86_64-cpython-310/tpm2_pytss._libtpm2_pytss.o -L/nix/store/xnh28i54p625r2xq6gaxqjysv75kk3ln-tpm2-tss-4.0.1/lib -L/nix/store/pkj7cgmz66assy7l18zc7j992npb41nx-python3-3.10.12/lib -ltss2-esys -ltss2-tctildr -ltss2-rc -ltss2-mu -ltss2-fapi -ltss2-policy -ltss2-tcti-spi-helper -o build/lib.linux-x86_64-cpython-310/tpm2_pytss/_libtpm2_pytss.abi3.so Traceback (most recent call last): File "/build/tpm2-pytss-2.1.0/nix_run_setup", line 8, in exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\\r\\n', '\\n'), __file__, 'exec')) File "setup.py", line 280, in setup( File "/nix/store/wyj4pwzaa11rlm3dkw4ywpl6l747g5sj-python3.10-setuptools-68.0.0/lib/python3.10/site-packages/setuptools/__init__.py", line 107, in setup return distutils.core.setup(**attrs) File "/nix/store/wyj4pwzaa11rlm3dkw4ywpl6l747g5sj-python3.10-setuptools-68.0.0/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup return run_commands(dist) File "/nix/store/wyj4pwzaa11rlm3dkw4ywpl6l747g5sj-python3.10-setuptools-68.0.0/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands dist.run_commands() File "/nix/store/wyj4pwzaa11rlm3dkw4ywpl6l747g5sj-python3.10-setuptools-68.0.0/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands self.run_command(cmd) File "/nix/store/wyj4pwzaa11rlm3dkw4ywpl6l747g5sj-python3.10-setuptools-68.0.0/lib/python3.10/site-packages/setuptools/dist.py", line 1234, in run_command super().run_command(command) File "/nix/store/wyj4pwzaa11rlm3dkw4ywpl6l747g5sj-python3.10-setuptools-68.0.0/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/nix/store/smfg2vy1f3c81m2jni503gb5kigq1hcl-python3.10-wheel-0.41.1/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 349, in run self.run_command("build") File "/nix/store/wyj4pwzaa11rlm3dkw4ywpl6l747g5sj-python3.10-setuptools-68.0.0/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command self.distribution.run_command(command) File "/nix/store/wyj4pwzaa11rlm3dkw4ywpl6l747g5sj-python3.10-setuptools-68.0.0/lib/python3.10/site-packages/setuptools/dist.py", line 1234, in run_command super().run_command(command) File "/nix/store/wyj4pwzaa11rlm3dkw4ywpl6l747g5sj-python3.10-setuptools-68.0.0/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/nix/store/wyj4pwzaa11rlm3dkw4ywpl6l747g5sj-python3.10-setuptools-68.0.0/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 131, in run self.run_command(cmd_name) File "/nix/store/wyj4pwzaa11rlm3dkw4ywpl6l747g5sj-python3.10-setuptools-68.0.0/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command self.distribution.run_command(command) File "/nix/store/wyj4pwzaa11rlm3dkw4ywpl6l747g5sj-python3.10-setuptools-68.0.0/lib/python3.10/site-packages/setuptools/dist.py", line 1234, in run_command super().run_command(command) File "/nix/store/wyj4pwzaa11rlm3dkw4ywpl6l747g5sj-python3.10-setuptools-68.0.0/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/nix/store/cx7dyi6hwhhs5a4k4blmndp36yinnvgh-python3.10-cffi-1.15.1/lib/python3.10/site-packages/cffi/setuptools_ext.py", line 144, in run base_class.run(self) File "setup.py", line 230, in run type_map, element_type_map = self.get_mappings() File "setup.py", line 210, in get_mappings past = parser.parse(pdata, "tss2_policy.h") File "/nix/store/i1h17ppzcs83lvkpl1vcass8hh37d8v9-python3.10-pycparser-2.21/lib/python3.10/site-packages/pycparser/c_parser.py", line 147, in parse return self.cparser.parse( File "/nix/store/i1h17ppzcs83lvkpl1vcass8hh37d8v9-python3.10-pycparser-2.21/lib/python3.10/site-packages/pycparser/ply/yacc.py", line 331, in parse return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc) File "/nix/store/i1h17ppzcs83lvkpl1vcass8hh37d8v9-python3.10-pycparser-2.21/lib/python3.10/site-packages/pycparser/ply/yacc.py", line 1199, in parseopt_notrack tok = call_errorfunc(self.errorfunc, errtoken, self) File "/nix/store/i1h17ppzcs83lvkpl1vcass8hh37d8v9-python3.10-pycparser-2.21/lib/python3.10/site-packages/pycparser/ply/yacc.py", line 193, in call_errorfunc r = errorfunc(token) File "/nix/store/i1h17ppzcs83lvkpl1vcass8hh37d8v9-python3.10-pycparser-2.21/lib/python3.10/site-packages/pycparser/c_parser.py", line 1931, in p_error self._parse_error( File "/nix/store/i1h17ppzcs83lvkpl1vcass8hh37d8v9-python3.10-pycparser-2.21/lib/python3.10/site-packages/pycparser/plyparser.py", line 67, in _parse_error raise ParseError("%s: %s" % (coord, msg)) pycparser.plyparser.ParseError: /nix/store/jjwhp4g4myz8mcbyjxnxspwf9d2rinjg-glibc-2.37-8-dev/include/bits/poll2.h:26:78: before: __asm__ /nix/store/1qxkj91n172qnypdrqdybh4p203nax7g-stdenv-linux/setup: line 1596: pop_var_context: head of shell_variables not a function context ```

Additional context

Seems to be the same error as #244107, but the current nixos-unstable (as of a999c1cc0c9eb2095729d5aa03e0d8f7ed256780) has the supposed fix (#245139). Maybe one of the other patches added since then (108c1d886668fd66c05176cf505ef5bdeadffbde 5933731483ec3b598fa016f6e3c50cd6c3e0a7e0) re-introduced the build failure?

Notify maintainers

@baloo

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix run nixpkgs#nix-info -- -m
 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.47, NixOS, 23.11 (Tapir), 23.11.20230827.a999c1c`
 - multi-user?: `no`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.17.0`
 - nixpkgs: `not found`
spacekitteh commented 1 year ago

I can confirm this failure

accelbread commented 1 year ago

I am also hitting this issue.

Both of the above mentioned commits build for me.

accelbread commented 1 year ago

Build failure was introduced with f900e733c1602b4c46064a9ef0514bb2345a8949

accelbread commented 1 year ago

From the commits from that merge, narrowed it down to 602e1b23925bee87d3662b56c8041f8065a7e468

accelbread commented 1 year ago

Followed merges back up to ed89997775d0e418e582e8eabbe121e7169c602a, though both parents have the failure there... :(

god464 commented 1 year ago

I have this issue too. This is my error message: https://fars.ee/b83l/sh This is the full log: https://fars.ee/S0iD

Cryolitia commented 1 year ago

I have the problem, too

lugeha commented 1 year ago

I don't have a patch for it yet, but adding -D_FORTIFY_SOURCE=0 to this list gets it to build.

https://github.com/baloo/tpm2-pytss/blob/89a24a0e2f02b51cb989e17df0b3229576730778/setup.py#L187

ezracelli commented 1 year ago

@lugeha Unless I'm mistaken, that was the fix applied by #245139 — first patch in the list here: https://github.com/NixOS/nixpkgs/blob/nixpkgs-unstable/pkgs/development/python-modules/tpm2-pytss/default.nix

https://github.com/NixOS/nixpkgs/blob/cddebdb60de376c1bdb7a4e6ee3d98355453fe56/pkgs/development/python-modules/tpm2-pytss/default.nix#L33-L43

Cryolitia commented 1 year ago

@lugeha Unless I'm mistaken, that was the fix applied by #245139 — first patch in the list here: https://github.com/NixOS/nixpkgs/blob/nixpkgs-unstable/pkgs/development/python-modules/tpm2-pytss/default.nix

https://github.com/NixOS/nixpkgs/blob/cddebdb60de376c1bdb7a4e6ee3d98355453fe56/pkgs/development/python-modules/tpm2-pytss/default.nix#L33-L43

It only add on L207, not on L187

Cryolitia commented 1 year ago

I don't have a patch for it yet, but adding -D_FORTIFY_SOURCE=0 to this list gets it to build.

https://github.com/baloo/tpm2-pytss/blob/89a24a0e2f02b51cb989e17df0b3229576730778/setup.py#L187

[deleted] Tried to make a patch file https://github.com/Cryolitia/tpm2-pytss/compare/9f27d0c0bdd42574b158c0372213e259fad80798..8a51a09fe998f147149ef77aed63ff6942ba929c.patch , but still not works for me

lugeha commented 1 year ago

Yeah, the overlay I build isn't working for me either. Not sure where I was that made it work...

final: prev: {
  python310 = prev.python310.override {
    packageOverrides = final: prev: {
      tpm2-pytss = prev.tpm2-pytss.overridePythonAttrs (old: {
        patches = [(prev.pkgs.writeText "fortify.patch" ''
--- a/setup.py
+++ b/setup.py
@@ -184,7 +184,7 @@ class type_generator(build_ext):
                 f"unable to find tss2_tpm2_types.h in {pk['include_dirs']}"
             )
         pdata = preprocess_file(
-            header_path, cpp_args=["-D__extension__=", "-D__attribute__(x)="]
+            header_path, cpp_args=["-D_FORTIFY_SOURCE=0", "-D__extension__=", "-D__attribute__(x)="]
         )
         parser = c_parser.CParser()
         ast = parser.parse(pdata, "tss2_tpm2_types.h")
        '')] ++ old.patches;
      });
    };
  };
}
Cryolitia commented 1 year ago

@lugeha find out it and u will be the hero hhh

accelbread commented 1 year ago

Ok, found the exact PR causing the issue: https://github.com/NixOS/nixpkgs/pull/246244

accelbread commented 1 year ago

The issue is that nixpkgs now forces the fortify setting, ignoring whats set in project's files.

So old method of disabling fortify for just one invocation is not enough

I've been trying to use hardeningDisable = [ "fortify" ]; and then setting fortify settings in the environment, but that is not working. With hardeningDisable = [ "fortify" ], the package builds, but with that set, it seems that trying to enable fortification in the environment has no effect. So not sure how to enable fortify for the built object files.

accelbread commented 1 year ago

The overlay I'm using until there is a better fix is:

final: prev: {
  python3 = prev.python3.override {
    packageOverrides = _: prev: {
      # TODO: Remove when fixed in nixpkgs
      tpm2-pytss = prev.tpm2-pytss.overridePythonAttrs {
        hardeningDisable = [ "fortify" ];
      };
    };
  };
}
baloo commented 1 year ago

Sorry for the lag, I'm out in holidays with family without much internet. I posted a fix in #252525 but I'm not sure I'll be able to follow up on it.