Environment variable with a hyphen is dropped in ubuntu

[mserdarsanli]

Feel free to close this as invalid, the bug is likely not in ninja, as it looks like ninja simply does a posix_spawn to launch a process without modifying the environment. Though interestingly I could not reproduce the issue without ninja (tried my own wrapper that does posix_spawn etc), so opening it here as someone else might have experienced the same issue.

populate buid.ninja with

rule environ
    command = env

build env.txt: environ

Run ninja with command:

env - FOO_bar=1 FOO_baz-baz=2 ninja

Expected output from env should include both FOO_bar and FOO_baz-baz variables, but on some systems you will only see the variable without hyphen in the name, those with hyphen are dropped somewhere.

So far I've tested: Ubuntu: Buggy Linux Mint: Buggy Fedora: Works Fine Arch: works Fine

I've also debugged by making ninja printing environ just before doing posix_spawn and I can see the variable FOO_baz-baz there, it just isn't visible in the launched process.

My guess is that ubuntu glibc does something wrong somewhere, but I couldn't find it

[eli-schwartz]

I can simplify the problem for you:

$ env - FOO_bar=1 FOO_baz-baz=2 bash -c env
FOO_baz-baz=2
PWD=/home/eschwartz/git/ninja
SHLVL=0
FOO_bar=1
_=/usr/bin/env
$ env - FOO_bar=1 FOO_baz-baz=2 dash -c env
PWD=/home/eschwartz/git/ninja
FOO_bar=1
$ env - FOO_bar=1 FOO_baz-baz=2 busybox sh -c env
SHLVL=1
FOO_baz-baz=2
PWD=/home/eschwartz/git/ninja
FOO_bar=1

Command lines in ninja are executed using sh -c, shells are not required to support environment variables whose names contain a hyphen -- that's not a portable name. Ninja preserves the variable just fine, until the shell itself drops it. Well, some shells anyway.

[mserdarsanli]

Thanks, that explains it. I my system sh is dash so it strips those variables. Ninja seems to hardcode /bin/sh path there so it can't simply be worked around.

shells are not required to support environment variables whose names contain a hyphen -- that's not a portable name.

Is this really the case? I can't find much about it. Looking at dash commit for the change, it seems to be added to fix the export use case, while bash instead keeps such env vars and not print them on export.

[eli-schwartz]

POSIX explicitly states that:

• environment variables containing uppercase letters, digits (as long as the first character isn't a digit) and underscores are portable names that are available to POSIX utilities
• environment variables containing the above, and additionally lowercase letters, constitutes a reserved namespace carved out by POSIX for the use of applications
• all other characters constitute a vendor extension, and vendors may feel free to permit it or reject it and neither choice is valid or invalid
• applications shall tolerate the act of seeing environment variables that do not conform to POSIX; such variables shall not crash the application or make it act improperly, compared to the state of affairs if that variable simply didn't exist, but no other guarantee is made other than "tolerate its presence"
• a shell is explicitly allowed to delete environment variables at startup, and not pass them to subprocesses, if they aren't used to initialize shell variables because they aren't a valid name (i.e they aren't part of the posix namespace for utility or application environment variables). This behavior is explicitly described, called out, and stated to be "unspecified".

Bash is conformant to the specification; when faced with non-portable environment variables that it doesn't understand and cannot make into shell variables, it deletes them before forking subprocesses and proceeds normally.

Dash is conformant the same way.

BusyBox sh is also conformant, but a different way; it chooses to preserve them for subprocesses, as it is permitted to do.