Open jgeerds opened 3 years ago
CC: @ony @edolstra
Hopefully https://github.com/NixOS/nix/pull/4831 addresses it.
@jgeerds , thank you for such details. I'm a bit confused. If I read it correctly, we get issue and at the same time we are missing extra slash at beginning.
As per POSIX spec //
should be equivalent to /
. I'm new to Nix derivations, but I think there should be something else beside those env-vars
.
Very first error looks like:
clang \
# ... nix store and defines
-I/private/tmp/nix-build-python3.8-Pillow-8.2.0debug.drv-1/Pillow-6.1.0/src/libImaging \
# ... nix store
-c src/_imaging.c -o build/temp.macosx-10.6-x86_64-3.8/src/_imaging.o
src/_webp.c:3:10: fatal error: 'Imaging.h' file not found
Is it possible to check contents of that /private/tmp/nix-build-python3.8-Pillow-8.2.0debug.drv-1/Pillow-6.1.0/src/libImaging
? (mostly interesting if it is completely empty or partially)
I guess output of nix show-derivation /nix/store/3jhxv9l1xnxbi85mb59g23dx8h9whpaw-python3.8-Pillow-8.2.0debug.drv
will be even more useful.
It looks like libImaging has Imaging.h. Could be something with how Python's C bindings resolve.
It looks like https://github.com/NixOS/nix/pull/4831 fixes it though - I no longer get fatal error: 'Imaging.h' file not found
, but do still get a failure in running the tests though.
It really weird because as soon as clang
being started Nix should not be involved, as I understand. If it would be a problem with incorrect paths, it should be visible either in arguments, or in environment (e.g. TMPDIR
) or in some file content produced by Nix (including generated paths in symlinks).
If -I...
provided with path to file it should be included into search path for header files and thus should be found. I was totally sure that there would be no Imaging.h
file in that path /private/tmp/.../libImaging
.
I was expecting some races between access to that file and how that file appears in that folder due to generation of its content in parallel or disappears due to some /tmp
clean-up activity kicked in after unpack.
Anyway. If #4831 fixes this issue - good :+1:
P.S. I also get test failures later. P.P.S.
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -I/nix/store/0qf4xb8zr6nf4xrjin7drpij5x2rvg8b-libwebp-1.1.0/include -I/nix/store/k2s0na7fmifi4cb9r0a634347s61y8as-libxcb-1.14-dev/include -fPIC -DHAVE_WEBPMUX -I/nix/store/bpzgxz0g1150vzw5nv2xzvch679dx256-freetype-2.10.4-dev/include -I/nix/store/6682psmzgmlssplgymdpi2rp2lrjf4n3-openjpeg-2.4.0-dev/include/openjpeg-2.4 -I/build/Pillow-6.1.0/src/libImaging -I/nix/store/xifvcap1gn5b5pdr5m6n5j669l1zq217-libjpeg-turbo-2.0.6-dev/include -I/nix/store/6682psmzgmlssplgymdpi2rp2lrjf4n3-openjpeg-2.4.0-dev/include -I/nix/store/pdskgbvrad17nsy15iz5z8icp44cs260-libtiff-4.2.0-dev/include -I/nix/store/bx0y0scv9dc56wk1zz9yi14lkc5js0p4-zlib-1.2.11-dev/include -I/nix/store/j2ysvcdqi40ksdb0cmv1jk3n10zhqp4v-lcms2-2.12-dev/include -I/nix/store/04nbcchky8zngrcbz92b2sr7ffbvl14f-libimagequant-2.14.1/include -I/nix/store/0qf4xb8zr6nf4xrjin7drpij5x2rvg8b-libwebp-1.1.0/include -I/nix/store/k2s0na7fmifi4cb9r0a634347s61y8as-libxcb-1.14-dev/include -I/nix/store/sfzsy9qa9w1656plqwhdfh3qvswgxdvk-python3-3.7.10/include -I/nix/store/sfzsy9qa9w1656plqwhdfh3qvswgxdvk-python3-3.7.10/include/python3.7m -c src/_webp.c -o build/temp.linux-x86_64-3.7/src/_webp.o
In my case on Linux NixOS 20.09 path is -I/build/Pillow-6.1.0/src/libImaging
which I believe because Linux supports filesystem view per process tree.
In yet another part of POSIX spec says:
If a pathname begins with two successive \<slash> characters, the first component following the leading \<slash> characters may be interpreted in an implementation-defined manner, although more than two leading \<slash> characters shall be treated as a single \<slash> character.
So there might be indeed special treatment of double-slashes in Darwin. Then maybe this issue can be resolved in favor of fix #4831
I marked this as stale due to inactivity. → More info
Describe the bug
Since
2.4pre20210503_6d2553a
I can no longer build a (modified) Version ofpython3Packages.pillow
on Darwin:The
Imaging.h
is provided by the same package. To track down the issue, I've build the derivation with each nightly version of Nix. 709a60a is the last version were the build succeeds (i.e. fails with 76980a1). This is the diff between the versions.Then I built 76980a1 myself and cherry-picked f3f228700a52857fe6e8632df4e935551ea219ff and it seems that this commit is the reason.
Please note that this issue only affects Darwin.
Steps To Reproduce
nix --experimental-features "nix-command flakes" shell github:NixOS/nix/76980a1f3daf6d2890f7686bfc8cdf6a8b9e8dae -c nix-build -K -E 'with import (builtins.fetchTarball https://github.com/NixOS/nixpkgs/archive/9775b39fd45a61c2a53aa6a90485fac8f87ce7c6.tar.gz) { }; python37Packages.pillow.overrideAttrs (old: { name = old.name + "debug"; src = python37Packages.fetchPypi { pname = "Pillow"; version = "6.1.0"; sha256 = "sha256-CAT3fLHpttvTdgHO4RKDu6OajUS53bBTQAxY4MDX2d4="; }; } )'
Replace
76980a1f3daf6d2890f7686bfc8cdf6a8b9e8dae
with709a60a045688cc5921574155958af9bc2349d02
and the build should work.Expected behavior
The build should succeed.
Additional context
(macOS Big Sur)
Nix has been installed as single user (FileVault enabled).
env-vars
diff between a failed and a successful build:Note the leading
/
vs//
/tmp/
is a symlink: