jgeerds
commented
3 years ago CC: @ony @edolstra
Open jgeerds opened 3 years ago
jgeerds
commented
3 years ago CC: @ony @edolstra
matthewbauer
commented
3 years ago Hopefully https://github.com/NixOS/nix/pull/4831 addresses it.
ony
commented
3 years ago @jgeerds , thank you for such details. I'm a bit confused. If I read it correctly, we get issue and at the same time we are missing extra slash at beginning.
As per POSIX spec // should be equivalent to /. I'm new to Nix derivations, but I think there should be something else beside those env-vars.
Very first error looks like:
clang \
# ... nix store and defines
-I/private/tmp/nix-build-python3.8-Pillow-8.2.0debug.drv-1/Pillow-6.1.0/src/libImaging \
# ... nix store
-c src/_imaging.c -o build/temp.macosx-10.6-x86_64-3.8/src/_imaging.o
src/_webp.c:3:10: fatal error: 'Imaging.h' file not foundIs it possible to check contents of that /private/tmp/nix-build-python3.8-Pillow-8.2.0debug.drv-1/Pillow-6.1.0/src/libImaging? (mostly interesting if it is completely empty or partially)
I guess output of nix show-derivation /nix/store/3jhxv9l1xnxbi85mb59g23dx8h9whpaw-python3.8-Pillow-8.2.0debug.drv will be even more useful.
matthewbauer
commented
3 years ago It looks like libImaging has Imaging.h. Could be something with how Python's C bindings resolve.
It looks like https://github.com/NixOS/nix/pull/4831 fixes it though - I no longer get fatal error: 'Imaging.h' file not found, but do still get a failure in running the tests though.
ony
commented
3 years ago It really weird because as soon as clang being started Nix should not be involved, as I understand. If it would be a problem with incorrect paths, it should be visible either in arguments, or in environment (e.g. TMPDIR) or in some file content produced by Nix (including generated paths in symlinks).
If -I... provided with path to file it should be included into search path for header files and thus should be found. I was totally sure that there would be no Imaging.h file in that path /private/tmp/.../libImaging.
I was expecting some races between access to that file and how that file appears in that folder due to generation of its content in parallel or disappears due to some /tmp clean-up activity kicked in after unpack.
Anyway. If #4831 fixes this issue - good :+1:
P.S. I also get test failures later. P.P.S.
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -I/nix/store/0qf4xb8zr6nf4xrjin7drpij5x2rvg8b-libwebp-1.1.0/include -I/nix/store/k2s0na7fmifi4cb9r0a634347s61y8as-libxcb-1.14-dev/include -fPIC -DHAVE_WEBPMUX -I/nix/store/bpzgxz0g1150vzw5nv2xzvch679dx256-freetype-2.10.4-dev/include -I/nix/store/6682psmzgmlssplgymdpi2rp2lrjf4n3-openjpeg-2.4.0-dev/include/openjpeg-2.4 -I/build/Pillow-6.1.0/src/libImaging -I/nix/store/xifvcap1gn5b5pdr5m6n5j669l1zq217-libjpeg-turbo-2.0.6-dev/include -I/nix/store/6682psmzgmlssplgymdpi2rp2lrjf4n3-openjpeg-2.4.0-dev/include -I/nix/store/pdskgbvrad17nsy15iz5z8icp44cs260-libtiff-4.2.0-dev/include -I/nix/store/bx0y0scv9dc56wk1zz9yi14lkc5js0p4-zlib-1.2.11-dev/include -I/nix/store/j2ysvcdqi40ksdb0cmv1jk3n10zhqp4v-lcms2-2.12-dev/include -I/nix/store/04nbcchky8zngrcbz92b2sr7ffbvl14f-libimagequant-2.14.1/include -I/nix/store/0qf4xb8zr6nf4xrjin7drpij5x2rvg8b-libwebp-1.1.0/include -I/nix/store/k2s0na7fmifi4cb9r0a634347s61y8as-libxcb-1.14-dev/include -I/nix/store/sfzsy9qa9w1656plqwhdfh3qvswgxdvk-python3-3.7.10/include -I/nix/store/sfzsy9qa9w1656plqwhdfh3qvswgxdvk-python3-3.7.10/include/python3.7m -c src/_webp.c -o build/temp.linux-x86_64-3.7/src/_webp.oIn my case on Linux NixOS 20.09 path is -I/build/Pillow-6.1.0/src/libImaging which I believe because Linux supports filesystem view per process tree.
veprbl
commented
3 years ago ony
commented
3 years ago In yet another part of POSIX spec says:
If a pathname begins with two successive \<slash> characters, the first component following the leading \<slash> characters may be interpreted in an implementation-defined manner, although more than two leading \<slash> characters shall be treated as a single \<slash> character.
So there might be indeed special treatment of double-slashes in Darwin. Then maybe this issue can be resolved in favor of fix #4831
stale[bot]
commented
2 years ago I marked this as stale due to inactivity. → More info
Describe the bug
Since
2.4pre20210503_6d2553aI can no longer build a (modified) Version ofpython3Packages.pillowon Darwin:The
Imaging.his provided by the same package. To track down the issue, I've build the derivation with each nightly version of Nix. 709a60a is the last version were the build succeeds (i.e. fails with 76980a1). This is the diff between the versions.Then I built 76980a1 myself and cherry-picked f3f228700a52857fe6e8632df4e935551ea219ff and it seems that this commit is the reason.
Please note that this issue only affects Darwin.
Steps To Reproduce
nix --experimental-features "nix-command flakes" shell github:NixOS/nix/76980a1f3daf6d2890f7686bfc8cdf6a8b9e8dae -c nix-build -K -E 'with import (builtins.fetchTarball https://github.com/NixOS/nixpkgs/archive/9775b39fd45a61c2a53aa6a90485fac8f87ce7c6.tar.gz) { }; python37Packages.pillow.overrideAttrs (old: { name = old.name + "debug"; src = python37Packages.fetchPypi { pname = "Pillow"; version = "6.1.0"; sha256 = "sha256-CAT3fLHpttvTdgHO4RKDu6OajUS53bBTQAxY4MDX2d4="; }; } )'Replace
76980a1f3daf6d2890f7686bfc8cdf6a8b9e8daewith709a60a045688cc5921574155958af9bc2349d02and the build should work.Expected behavior
The build should succeed.
Additional context
(macOS Big Sur)
Nix has been installed as single user (FileVault enabled).
env-varsdiff between a failed and a successful build:Note the leading
/vs///tmp/is a symlink: