oracle / graal

GraalVM compiles Java applications into native executables that start instantly, scale fast, and use fewer compute resources 🚀
https://www.graalvm.org
Other
20.34k stars 1.63k forks source link

GraalVM 23.0.0's native-image is cleaning the environment before calling C compiler #7502

Closed thiagokokada closed 10 months ago

thiagokokada commented 1 year ago

Describe the issue I am working in bumping GraalVM package to the https://github.com/graalvm/graalvm-ce-builds/releases/tag/jdk-21.0.0 inside nixpkgs (see PR) and bumped in the following issue.

Using the latest stable release, it seems native-image is cleaning the environment variables before calling the C compiler (cc). I did the following test by creating a small wrapper script for cc that basically does:

echo "=== Environment CC start ===" >> /tmp/environment
env >> /tmp/environment
echo "==== Environment CC end ====" >> /tmp/environment
${stdenv.cc}/bin/cc "$@"

And this is the result:

Builder environment (I removed a few things since Nix exports all the derivation attributes as environmental variables, so if I put everything here it would be confusing):

SHELL=/nix/store/dvk14kpsjv70jcqlj6z3izxwchka3ic3-bash-5.2-p15/bin/bash
NIX_BUILD_CORES=12
configureFlags=--prefix=/nix/store/l610yak0k8f7gnwbsniqwag9vfwyh3sc-graalvm-ce-21.0.0
NIX_CC_WRAPPER_TARGET_HOST_x86_64_apple_darwin=1
mesonFlags=
shell=/nix/store/dvk14kpsjv70jcqlj6z3izxwchka3ic3-bash-5.2-p15/bin/bash
depsHostHost=
__sandboxProfile=
STRINGS=strings
src=/nix/store/4ppdvvcrbhk72p9n2njbhnmkva7cwgp4-graalvm-community-jdk-21.0.0_macos-x64_bin.tar.gz
depsTargetTarget=
stdenv=/nix/store/7bpyxkinmlrwa8bn8pgi7177jnmjzlvj-stdenv-darwin
builder=/nix/store/dvk14kpsjv70jcqlj6z3izxwchka3ic3-bash-5.2-p15/bin/bash
MACOSX_DEPLOYMENT_TARGET=10.12
PWD=/private/tmp/nix-build-graalvm-ce-21.0.0.drv-0
SOURCE_DATE_EPOCH=315532800
pname=graalvm-ce
NIX_ENFORCE_NO_NATIVE=1
__propagatedSandboxProfile=
CXX=clang++
TEMPDIR=/private/tmp/nix-build-graalvm-ce-21.0.0.drv-0
system=x86_64-darwin
TZ=UTC
HOST_PATH=/nix/store/cf5x52z1vw9pxkwjrypxzjc006ivih3k-coreutils-9.3/bin:/nix/store/0idlnnlj3r5qvdi2c328qxhnasqlg7zr-findutils-4.9.0/bin:/nix/store/2y17vbm5wqss9hq0gwhx52z89x1sm3sl-diffutils-3.10/bin:/nix/store/z8y8ldjv16hx44kbhamm03i2jc43y5z3-gnused-4.9/bin:/nix/store/dh13glbrmfzysam1shbkakjjla9i51q1-gnugrep-3.11/bin:/nix/store/db8h92bj71zp9zy3npkya6c9n7vcimkz-gawk-5.2.2/bin:/nix/store/h786vhh9gw7c5zhq1pq8cxqgmzm5bgv7-gnutar-1.35/bin:/nix/store/ljk4vl7ja0pxznawh3jyxxma7vr09sh2-gzip-1.13/bin:/nix/store/d19x83q7ay3ybqzx1idgy8z19alxil6v-bzip2-1.0.8-bin/bin:/nix/store/wmgi910bjl21rc6bvfjl1hqhwdxijr98-gnumake-4.4.1/bin:/nix/store/dvk14kpsjv70jcqlj6z3izxwchka3ic3-bash-5.2-p15/bin:/nix/store/rwx19xxwfpbxck949zf9c8ysishvfvi3-patch-2.7.6/bin:/nix/store/7r4qr7w1szis7siyx5dcq22lsrzc9izp-xz-5.4.4-bin/bin:/nix/store/flc70ndgc8yxn6sn2y3xgy6dhy4a9hqm-file-5.45/bin
doInstallCheck=1
HOME=/homeless-shelter
NIX_BINTOOLS=/nix/store/a0sn94fkr24336ipljc5g9llcdhdif82-cctools-binutils-darwin-wrapper-11.1.0-973.0.1
GZIP_NO_TIMESTAMPS=1
NIX_DONT_SET_RPATH=1
depsTargetTargetPropagated=
cmakeFlags=
NIX_SSL_CERT_FILE=/no-cert-file.crt
version=21.0.0
LD_DYLD_PATH=/usr/lib/dyld
outputs=out
NIX_STORE=/nix/store
TMPDIR=/private/tmp/nix-build-graalvm-ce-21.0.0.drv-0
LD=ld
NIX_ENFORCE_PURITY=1
doCheck=
NIX_LOG_FD=2
NIX_DONT_SET_RPATH_FOR_BUILD=1
__propagatedImpureHostDeps=
depsBuildBuild=
TERM=xterm-256color
NIX_NO_SELF_RPATH=1
PATH_LOCALE=/nix/store/zw83nzk917f7idhydyhjhc62pnj9bh11-adv_cmds-119-locale/share/locale
SIZE=size
propagatedNativeBuildInputs=
strictDeps=
lt_cv_deplibs_check_method=pass_all
AR=ar
AS=as
TEMP=/private/tmp/nix-build-graalvm-ce-21.0.0.drv-0
SHLVL=1
NIX_BUILD_TOP=/private/tmp/nix-build-graalvm-ce-21.0.0.drv-0
NM=nm
NIX_CFLAGS_COMPILE=-F/nix/store/lv8wrxbbaysn9alvcqln5lck6m3s7f5p-apple-framework-CoreFoundation/Library/Frameworks  -frandom-seed=l610yak0k8 -Wno-deprecated-declarations -isystem /nix/store/mldj2ak4kffx9n952g4iskz6qf4286r0-libcxx-11.1.0-dev/include -isystem /nix/store/s64a0f88cc5s84kvhiavcx7v58q8sf08-libcxxabi-11.1.0-dev/include -isystem /nix/store/w2mjbr2v8hjk48a2hx9xy9d2jkizjvls-compiler-rt-libc-11.1.0-dev/include -isystem /nix/store/l45j5havxdqyvdqn0k8wqbl4lwhhafbl-zlib-1.3-dev/include -iframework /nix/store/4899s2h15zjd62s7859jm51ikkz78w3f-apple-framework-Foundation/Library/Frameworks -iframework /nix/store/ifhkixy53bjrfgvmzkw45x1ndkwlabbr-apple-framework-ApplicationServices/Library/Frameworks -iframework /nix/store/lw3d4i1hj248dwmsaw4fp7pyj5g2285z-apple-framework-CoreGraphics/Library/Frameworks -iframework /nix/store/lipj03xmx94wcqm1wcj9a3xpm2xijm0v-apple-framework-Accelerate/Library/Frameworks -iframework /nix/store/mzl9d31fvfmaymm2rvr2nyk0c3ihi7r1-apple-framework-CoreWLAN/Library/Frameworks -iframework /nix/store/02b5bzy909004izfz1fqgyy6wpmbhnd3-apple-framework-SecurityFoundation/Library/Frameworks -iframework /nix/store/9gky02pqq42rdvzs5ngfdcbgq11nk2l4-apple-framework-IOBluetooth/Library/Frameworks -iframework /nix/store/5x78mnnxy55c3rrsrg2pqdans8nklg78-apple-framework-CoreBluetooth/Library/Frameworks -iframework /nix/store/nbh46cp28lqf9jl7kcvs79ns2vz5pv0c-apple-framework-IOKit/Library/Frameworks -iframework /nix/store/0ws61795nrasbpqcqfllkmrib09pvahr-apple-framework-IOSurface/Library/Frameworks -isystem /nix/store/lhqr3wc9g059x7f75y7gbcj7pnwz6z7x-apple-lib-xpc/include -iframework /nix/store/4gf2mkagamg9f3837ri8ap9p1q8cmw5s-apple-framework-SystemConfiguration/Library/Frameworks -iframework /nix/store/cm1sxwdb9falxvcq5qx6j2s343f0ip5z-apple-framework-Security/Library/Frameworks -iframework /nix/store/7r2fc3fq0jj5vpi0ani9m8rbxv93gljn-apple-framework-CoreServices/Library/Frameworks -iframework /nix/store/3ij1bhi2lbm5lrg9qhjl2aqgwb0b2b90-apple-framework-CFNetwork/Library/Frameworks -iframework /nix/store/qmrd462dqk31dnh37s4hjyyyn67fj8lp-apple-framework-CoreAudio/Library/Frameworks -iframework /nix/store/462s88fknssvhxi6p8acxhiq7bns6bzv-apple-framework-CoreData/Library/Frameworks -iframework /nix/store/lv8wrxbbaysn9alvcqln5lck6m3s7f5p-apple-framework-CoreFoundation/Library/Frameworks -iframework /nix/store/wldc5cmfcv1slnqm70zgb25n76p9g377-apple-framework-DiskArbitration/Library/Frameworks -iframework /nix/store/h6hqsxp1wxg29f32dzsl0cnpp5p188bk-apple-framework-NetFS/Library/Frameworks -iframework /nix/store/af7s94n19b4r5sgmhz5h04b59gvkbnb9-apple-framework-OpenDirectory/Library/Frameworks -iframework /nix/store/4kvbdqlfcws6pbcwhqajfpg7ndnvk2jj-apple-framework-ServiceManagement/Library/Frameworks -iframework /nix/store/b6qxvfcmgy76ws18wclb05zyhlhhykng-apple-framework-CoreText/Library/Frameworks -iframework /nix/store/l977kr1dmdbnqx3v332man6p00iv22kk-apple-framework-ImageIO/Library/Frameworks -isystem /nix/store/vpimz1yfdsjkp77y1in55mdpbjaffczc-objc4-709.1/include -iframework /nix/store/7m03xjfdkq23dfazy6k1wr79ssz22qbp-swift-corefoundation-unstable-2018-09-14/Library/Frameworks -isystem /nix/store/mldj2ak4kffx9n952g4iskz6qf4286r0-libcxx-11.1.0-dev/include -isystem /nix/store/s64a0f88cc5s84kvhiavcx7v58q8sf08-libcxxabi-11.1.0-dev/include -isystem /nix/store/w2mjbr2v8hjk48a2hx9xy9d2jkizjvls-compiler-rt-libc-11.1.0-dev/include -isystem /nix/store/l45j5havxdqyvdqn0k8wqbl4lwhhafbl-zlib-1.3-dev/include -iframework /nix/store/4899s2h15zjd62s7859jm51ikkz78w3f-apple-framework-Foundation/Library/Frameworks -iframework /nix/store/ifhkixy53bjrfgvmzkw45x1ndkwlabbr-apple-framework-ApplicationServices/Library/Frameworks -iframework /nix/store/lw3d4i1hj248dwmsaw4fp7pyj5g2285z-apple-framework-CoreGraphics/Library/Frameworks -iframework /nix/store/lipj03xmx94wcqm1wcj9a3xpm2xijm0v-apple-framework-Accelerate/Library/Frameworks -iframework /nix/store/mzl9d31fvfmaymm2rvr2nyk0c3ihi7r1-apple-framework-CoreWLAN/Library/Frameworks -iframework /nix/store/02b5bzy909004izfz1fqgyy6wpmbhnd3-apple-framework-SecurityFoundation/Library/Frameworks -iframework /nix/store/9gky02pqq42rdvzs5ngfdcbgq11nk2l4-apple-framework-IOBluetooth/Library/Frameworks -iframework /nix/store/5x78mnnxy55c3rrsrg2pqdans8nklg78-apple-framework-CoreBluetooth/Library/Frameworks -iframework /nix/store/nbh46cp28lqf9jl7kcvs79ns2vz5pv0c-apple-framework-IOKit/Library/Frameworks -iframework /nix/store/0ws61795nrasbpqcqfllkmrib09pvahr-apple-framework-IOSurface/Library/Frameworks -isystem /nix/store/lhqr3wc9g059x7f75y7gbcj7pnwz6z7x-apple-lib-xpc/include -iframework /nix/store/4gf2mkagamg9f3837ri8ap9p1q8cmw5s-apple-framework-SystemConfiguration/Library/Frameworks -iframework /nix/store/cm1sxwdb9falxvcq5qx6j2s343f0ip5z-apple-framework-Security/Library/Frameworks -iframework /nix/store/7r2fc3fq0jj5vpi0ani9m8rbxv93gljn-apple-framework-CoreServices/Library/Frameworks -iframework /nix/store/3ij1bhi2lbm5lrg9qhjl2aqgwb0b2b90-apple-framework-CFNetwork/Library/Frameworks -iframework /nix/store/qmrd462dqk31dnh37s4hjyyyn67fj8lp-apple-framework-CoreAudio/Library/Frameworks -iframework /nix/store/462s88fknssvhxi6p8acxhiq7bns6bzv-apple-framework-CoreData/Library/Frameworks -iframework /nix/store/lv8wrxbbaysn9alvcqln5lck6m3s7f5p-apple-framework-CoreFoundation/Library/Frameworks -iframework /nix/store/wldc5cmfcv1slnqm70zgb25n76p9g377-apple-framework-DiskArbitration/Library/Frameworks -iframework /nix/store/h6hqsxp1wxg29f32dzsl0cnpp5p188bk-apple-framework-NetFS/Library/Frameworks -iframework /nix/store/af7s94n19b4r5sgmhz5h04b59gvkbnb9-apple-framework-OpenDirectory/Library/Frameworks -iframework /nix/store/4kvbdqlfcws6pbcwhqajfpg7ndnvk2jj-apple-framework-ServiceManagement/Library/Frameworks -iframework /nix/store/b6qxvfcmgy76ws18wclb05zyhlhhykng-apple-framework-CoreText/Library/Frameworks -iframework /nix/store/l977kr1dmdbnqx3v332man6p00iv22kk-apple-framework-ImageIO/Library/Frameworks -isystem /nix/store/vpimz1yfdsjkp77y1in55mdpbjaffczc-objc4-709.1/include -iframework /nix/store/7m03xjfdkq23dfazy6k1wr79ssz22qbp-swift-corefoundation-unstable-2018-09-14/Library/Frameworks
__impureHostDeps=/bin/sh /usr/lib/libSystem.B.dylib /usr/lib/system/libunc.dylib /dev/zero /dev/random /dev/urandom /bin/sh
NIX_BINTOOLS_WRAPPER_TARGET_HOST_x86_64_apple_darwin=1
patches=
preFixup=
NIX_IGNORE_LD_THROUGH_GCC=1
out=/nix/store/l610yak0k8f7gnwbsniqwag9vfwyh3sc-graalvm-ce-21.0.0
STRIP=strip
XDG_DATA_DIRS=/nix/store/n8rclk8s39lv147ka8hvlsmp459p7dqx-unzip-6.0/share
TMP=/private/tmp/nix-build-graalvm-ce-21.0.0.drv-0
PATH=/nix/store/n8rclk8s39lv147ka8hvlsmp459p7dqx-unzip-6.0/bin:/nix/store/adnfb3qi50jn2cqbp8g524rm6mxcwhz7-clang-wrapper-11.1.0/bin:/nix/store/rngczdw1717sqqp6hc0j3sm8szjracck-clang-11.1.0/bin:/nix/store/cf5x52z1vw9pxkwjrypxzjc006ivih3k-coreutils-9.3/bin:/nix/store/a0sn94fkr24336ipljc5g9llcdhdif82-cctools-binutils-darwin-wrapper-11.1.0-973.0.1/bin:/nix/store/zf13rw6lv61gbamlfcf4bn75wfi7l78v-cctools-binutils-darwin-11.1.0-973.0.1/bin:/nix/store/cf5x52z1vw9pxkwjrypxzjc006ivih3k-coreutils-9.3/bin:/nix/store/0idlnnlj3r5qvdi2c328qxhnasqlg7zr-findutils-4.9.0/bin:/nix/store/2y17vbm5wqss9hq0gwhx52z89x1sm3sl-diffutils-3.10/bin:/nix/store/z8y8ldjv16hx44kbhamm03i2jc43y5z3-gnused-4.9/bin:/nix/store/dh13glbrmfzysam1shbkakjjla9i51q1-gnugrep-3.11/bin:/nix/store/db8h92bj71zp9zy3npkya6c9n7vcimkz-gawk-5.2.2/bin:/nix/store/h786vhh9gw7c5zhq1pq8cxqgmzm5bgv7-gnutar-1.35/bin:/nix/store/ljk4vl7ja0pxznawh3jyxxma7vr09sh2-gzip-1.13/bin:/nix/store/d19x83q7ay3ybqzx1idgy8z19alxil6v-bzip2-1.0.8-bin/bin:/nix/store/wmgi910bjl21rc6bvfjl1hqhwdxijr98-gnumake-4.4.1/bin:/nix/store/dvk14kpsjv70jcqlj6z3izxwchka3ic3-bash-5.2-p15/bin:/nix/store/rwx19xxwfpbxck949zf9c8ysishvfvi3-patch-2.7.6/bin:/nix/store/7r4qr7w1szis7siyx5dcq22lsrzc9izp-xz-5.4.4-bin/bin:/nix/store/flc70ndgc8yxn6sn2y3xgy6dhy4a9hqm-file-5.45/bin
propagatedBuildInputs=/nix/store/b9lnskajdhzygbjycv3yivm0pab8rhqc-set-java-classpath-hook /nix/store/l45j5havxdqyvdqn0k8wqbl4lwhhafbl-zlib-1.3-dev /nix/store/4899s2h15zjd62s7859jm51ikkz78w3f-apple-framework-Foundation
CC=clang
NIX_CC=/nix/store/adnfb3qi50jn2cqbp8g524rm6mxcwhz7-clang-wrapper-11.1.0
NIX_COREFOUNDATION_RPATH=/System/Library/Frameworks
depsBuildTargetPropagated=
depsBuildBuildPropagated=
CONFIG_SHELL=/nix/store/dvk14kpsjv70jcqlj6z3izxwchka3ic3-bash-5.2-p15/bin/bash
__structuredAttrs=
RANLIB=ranlib
NIX_HARDENING_ENABLE=fortify fortify3 stackprotector pic strictoverflow format relro bindnow
__darwinAllowLocalNetworking=
OLDPWD=/private/tmp/nix-build-graalvm-ce-21.0.0.drv-0
NIX_LDFLAGS= -L/nix/store/w5r2hmvjn590hc8lnz6bxj3mf6vz83gf-libcxx-11.1.0/lib -L/nix/store/gc032163z7x8gnyl8l5q2846a8r3n67p-libcxxabi-11.1.0/lib -L/nix/store/9isl4d07rwzb543mlrli08pshv2vng9p-compiler-rt-libc-11.1.0/lib -L/nix/store/l7sa6ddym6n2dxah158vbvkzcf7m7abg-zlib-1.3/lib -L/nix/store/vpimz1yfdsjkp77y1in55mdpbjaffczc-objc4-709.1/lib -L/nix/store/w5r2hmvjn590hc8lnz6bxj3mf6vz83gf-libcxx-11.1.0/lib -L/nix/store/gc032163z7x8gnyl8l5q2846a8r3n67p-libcxxabi-11.1.0/lib -L/nix/store/9isl4d07rwzb543mlrli08pshv2vng9p-compiler-rt-libc-11.1.0/lib -L/nix/store/l7sa6ddym6n2dxah158vbvkzcf7m7abg-zlib-1.3/lib -L/nix/store/vpimz1yfdsjkp77y1in55mdpbjaffczc-objc4-709.1/lib /nix/store/lv8wrxbbaysn9alvcqln5lck6m3s7f5p-apple-framework-CoreFoundation/Library/Frameworks/CoreFoundation.framework/CoreFoundation.tbd
nativeBuildInputs=/nix/store/n8rclk8s39lv147ka8hvlsmp459p7dqx-unzip-6.0 /nix/store/13v2iwmlhk1pvnm9j4b0ma108lrab48s-make-shell-wrapper-hook
name=graalvm-ce-21.0.0
depsHostHostPropagated=
_=/nix/store/cf5x52z1vw9pxkwjrypxzjc006ivih3k-coreutils-9.3/bin/env

CC environment:

USE_NATIVE_IMAGE_JAVA_PLATFORM_MODULE_SYSTEM=true
PWD=/private/var/folders/zf/6dxm00pn1xx19sg7cbkk4dqm00009d/T/SVM-6586594021044324100
HOME=/homeless-shelter
SHLVL=1
PATH=/nix/store/6llivcfh72vxwcmd9v3bbhdnli1vvpiy-cc/bin:/nix/store/adnfb3qi50jn2cqbp8g524rm6mxcwhz7-clang-wrapper-11.1.0/bin:/nix/store/n8rclk8s39lv147ka8hvlsmp459p7dqx-unzip-6.0/bin:/nix/store/rngczdw1717sqqp6hc0j3sm8szjracck-clang-11.1.0/bin:/nix/store/cf5x52z1vw9pxkwjrypxzjc006ivih3k-coreutils-9.3/bin:/nix/store/a0sn94fkr24336ipljc5g9llcdhdif82-cctools-binutils-darwin-wrapper-11.1.0-973.0.1/bin:/nix/store/zf13rw6lv61gbamlfcf4bn75wfi7l78v-cctools-binutils-darwin-11.1.0-973.0.1/bin:/nix/store/cf5x52z1vw9pxkwjrypxzjc006ivih3k-coreutils-9.3/bin:/nix/store/0idlnnlj3r5qvdi2c328qxhnasqlg7zr-findutils-4.9.0/bin:/nix/store/2y17vbm5wqss9hq0gwhx52z89x1sm3sl-diffutils-3.10/bin:/nix/store/z8y8ldjv16hx44kbhamm03i2jc43y5z3-gnused-4.9/bin:/nix/store/dh13glbrmfzysam1shbkakjjla9i51q1-gnugrep-3.11/bin:/nix/store/db8h92bj71zp9zy3npkya6c9n7vcimkz-gawk-5.2.2/bin:/nix/store/h786vhh9gw7c5zhq1pq8cxqgmzm5bgv7-gnutar-1.35/bin:/nix/store/ljk4vl7ja0pxznawh3jyxxma7vr09sh2-gzip-1.13/bin:/nix/store/d19x83q7ay3ybqzx1idgy8z19alxil6v-bzip2-1.0.8-bin/bin:/nix/store/wmgi910bjl21rc6bvfjl1hqhwdxijr98-gnumake-4.4.1/bin:/nix/store/dvk14kpsjv70jcqlj6z3izxwchka3ic3-bash-5.2-p15/bin:/nix/store/rwx19xxwfpbxck949zf9c8ysishvfvi3-patch-2.7.6/bin:/nix/store/7r4qr7w1szis7siyx5dcq22lsrzc9izp-xz-5.4.4-bin/bin:/nix/store/flc70ndgc8yxn6sn2y3xgy6dhy4a9hqm-file-5.45/bin
__CF_USER_TEXT_ENCODING=0x12D:0:0
_=/nix/store/cf5x52z1vw9pxkwjrypxzjc006ivih3k-coreutils-9.3/bin/env

Why is this an issue? In Nix we don't have the libraries in their expected places, so we need to give some help for the compiler to find everything. We do this by wrapping the C compiler and exporting some environmental variables that allows the compiler to figure out where everything is. For example NIX_CFLAGS_COMPILE and NIX_LDFLAGS. By removing those environment variables the compiler can't find the critical libraries, resulting in the following issue we are seeing:

========================================================================================================================
GraalVM Native Image: Generating 'helloworld' (executable)...
========================================================================================================================
For detailed information and explanations on the build output, visit:
https://github.com/oracle/graal/blob/master/docs/reference-manual/native-image/BuildOutput.md
------------------------------------------------------------------------------------------------------------------------

[1/8] Initializing...                                                                                    (0.0s @ 0.09GB)
Error: Error compiling query code (in /var/folders/zf/6dxm00pn1xx19sg7cbkk4dqm00009d/T/SVM-6586594021044324100/PosixDirectives.c). Compiler command '/nix/store/6llivcfh72vxwcmd9v3bbhdnli1vvpiy-cc/bin/cc -Wall -Werror -Wno-tautological-compare -ObjC -o /var/folders/zf/6dxm00pn1xx19sg7cbkk4dqm00009d/T/SVM-6586594021044324100/PosixDirectives /var/folders/zf/6dxm00pn1xx19sg7cbkk4dqm00009d/T/SVM-6586594021044324100/PosixDirectives.c' output included error: /var/folders/zf/6dxm00pn1xx19sg7cbkk4dqm00009d/T/SVM-6586594021044324100/PosixDirectives.c:34:10: fatal error: 'Foundation/Foundation.h' file not found
    C file contents around line 34:
    /var/folders/zf/6dxm00pn1xx19sg7cbkk4dqm00009d/T/SVM-6586594021044324100/PosixDirectives.c:33: #include <unistd.h>
    /var/folders/zf/6dxm00pn1xx19sg7cbkk4dqm00009d/T/SVM-6586594021044324100/PosixDirectives.c:34: #include <Foundation/Foundation.h>
    /var/folders/zf/6dxm00pn1xx19sg7cbkk4dqm00009d/T/SVM-6586594021044324100/PosixDirectives.c:35: #include <mach/mach.h>
com.oracle.svm.core.util.UserError$UserException: Error compiling query code (in /var/folders/zf/6dxm00pn1xx19sg7cbkk4dqm00009d/T/SVM-6586594021044324100/PosixDirectives.c). Compiler command '/nix/store/6llivcfh72vxwcmd9v3bbhdnli1vvpiy-cc/bin/cc -Wall -Werror -Wno-tautological-compare -ObjC -o /var/folders/zf/6dxm00pn1xx19sg7cbkk4dqm00009d/T/SVM-6586594021044324100/PosixDirectives /var/folders/zf/6dxm00pn1xx19sg7cbkk4dqm00009d/T/SVM-6586594021044324100/PosixDirectives.c' output included error: /var/folders/zf/6dxm00pn1xx19sg7cbkk4dqm00009d/T/SVM-6586594021044324100/PosixDirectives.c:34:10: fatal error: 'Foundation/Foundation.h' file not found
    C file contents around line 34:
    /var/folders/zf/6dxm00pn1xx19sg7cbkk4dqm00009d/T/SVM-6586594021044324100/PosixDirectives.c:33: #include <unistd.h>
    /var/folders/zf/6dxm00pn1xx19sg7cbkk4dqm00009d/T/SVM-6586594021044324100/PosixDirectives.c:34: #include <Foundation/Foundation.h>
    /var/folders/zf/6dxm00pn1xx19sg7cbkk4dqm00009d/T/SVM-6586594021044324100/PosixDirectives.c:35: #include <mach/mach.h>
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.util.UserError.abort(UserError.java:126)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.c.NativeLibraries.reportErrors(NativeLibraries.java:380)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.processNativeLibraryImports(NativeImageGenerator.java:1751)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.setupNativeLibraries(NativeImageGenerator.java:1198)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.setupNativeImage(NativeImageGenerator.java:961)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:591)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:551)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:538)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:720)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.start(NativeImageGeneratorRunner.java:142)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:97)
------------------------------------------------------------------------------------------------------------------------
                         0.3s (3.1% of total time) in 7 GCs | Peak RSS: 0.41GB | CPU load: 1.17
========================================================================================================================
Finished generating 'helloworld' in 7.9s.

As you can see, Foundation/Foundation.h is not found (this build was done in an x86_64 MacOS environment). If you look at the NIX_COMPILE_FLAGS and NIX_LDFLAGS, those includes the incantations that the C compiler needs to add to find the libraries. I created a workaround where we re-inject the expected NIX_COMPILE_FLAGS and NIX_LDFLAGS inside a cc wrapper, and this was sufficient to get it to build again. However this is ugly and fragile. Also, we didn't need any of those things until GraalVM 22.3.1, so I imagine this is a recent change done in the 23.0.0 release.

If the environment cleaning in the C compiler is proposital to workaround some other issue, can we have some flag to disable this behavior or at least some way were we can add environment variables to the allowlist (so they would be re-added to the C compiler environment)? Keep in mind that we always run a build inside a proper sandbox, so the issue where some random environment variables is breaking builds (something that I imagine was the reason for this change) wouldn't affect us.

Steps to reproduce the issue

  1. Create a cc wrapper similar to the one below, name this as cc and put it before anything else in your PATH
    echo "=== Environment CC start ===" >> /tmp/environment
    env >> /tmp/environment
    echo "==== Environment CC end ====" >> /tmp/environment
    /usr/bin/cc "$@"
  2. Compile something with native-image
  3. cat /tmp/environment and compare with your current environment. You will see that it is missing tons of environment variables

Describe GraalVM and your environment:

More details N/A

oubidar-Abderrahim commented 1 year ago

Thank you for reporting this, we'll take a look into it shortly

oubidar-Abderrahim commented 1 year ago

Hi,

Could you please provide more specific steps to reproduce the issue? I've created the CC file but it wasn't executed, there is no file under /tmp/environment

thiagokokada commented 1 year ago

Hi,

Could you please provide more specific steps to reproduce the issue? I've created the CC file but it wasn't executed, there is no file under /tmp/environment

You need to create a cc script that will print the environment in that file. You can do something like this:

mkdir -p ~/bin
export PATH=$HOME/bin:$PATH

Create a new file in ~/bin/cc and paste this contents:

echo "=== Environment CC start ===" >> /tmp/environment
env >> /tmp/environment
echo "==== Environment CC end ====" >> /tmp/environment
/usr/bin/cc "$@"
oubidar-Abderrahim commented 1 year ago

Unfortunately, nothing changed here are the steps I'm following: Screen Shot 2023-10-24 at 16 02 36 Screen Shot 2023-10-24 at 16 03 49

If there are any missing steps please include them in your description. Thank you

thiagokokada commented 1 year ago

I think the script is missing the execution bit:

$ chmod +x $HOME/bin/cc

This should work.

thiagokokada commented 1 year ago

Just to make sure, before running GraalVM, run cc and do a cat /tmp/environment to see if it is working correctly.

thiagokokada commented 11 months ago

@oubidar-Abderrahim Any updates on this issue? I have a very strong suspicious this is also affecting Linux systems.

oubidar-Abderrahim commented 11 months ago

Unfortunately, I cannot reproduce the issue. after building the image I compared the Env variable in /tmp/environment and I do see some extra one after building the image, but nothing is missing

borkdude commented 10 months ago

Maybe related? https://github.com/oracle/graal/issues/7981

oubidar-Abderrahim commented 10 months ago

I'll close this issue for now as not reproducible. If the suggestion from #7981 is not solving the issue, and you can provide a more reliable reproducer, please reopen or create a new issue for it and we'll check it again. Thank you

thiagokokada commented 10 months ago

Thanks @oubidar-Abderrahim, using export NATIVE_IMAGE_DEPRECATED_BUILDER_SANITATION=true did the trick for now.

Do we have any idea for how long this environment variable will be supported?

oubidar-Abderrahim commented 10 months ago

We don't have a specific date yet, but I would suggest using -E<env-var-key>[=<env-var-value>] instead so that you won't be affected once it gets removed

borkdude commented 10 months ago

Would it be possible to add an env var wildcard? I have like 20+ environment variables that I'm capturing at build time starting with BABASHKA_

thiagokokada commented 10 months ago

Would it be possible to add an env var wildcard? I have like 20+ environment variables that I'm capturing at build time starting with BABASHKA_

We also need to inject a bunch of environment variables.