devbox global puts nixpkgs stdenv in my PATH

gcurtis commented 9 months ago

Current Behavior (bug)

Running eval "$(devbox global shellenv)" adds the nixpkgs stdenv packages to my PATH, even though those packages weren't added with devbox global add. This includes packages like:

clang
bash
coreutils

which can have unintentional (and surprising) effects on the rest of the system.

For example, the clang wrapper sets flags that modify the system frameworks/libraries that get linked by default, which can break compiling macOS programs. For example, the linker fails to find the macOS Virtualization Framework with:

$ clang -framework Foundation -framework Virtualization -framework Cocoa ...
fatal error: 'Virtualization/Virtualization.h' file not found
#import <Virtualization/Virtualization.h>
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3 warnings and 1 error generated.

Running the same exact command with /usr/bin/clang succeeds.

This also happens with a non-global devbox shell, making it difficult to use Devbox with native macOS development.

Expected Behavior (fix)

Don't add packages to my PATH unless I've explicitly added it to my devbox global profile. This might mean not using the print-dev-env environment for global and instead only adding the nix profile's bin directory to the PATH.

One thing to be aware of is that these packages might be needed so that other packages work as expected. For example, python/pip might need clang in order to compile native extensions. We should test this out to better understand the effects.

Packages in ~/.local/share/devbox/global/default:

{
  "packages": [
    "awscli2@latest",
    "bash@latest",
    "brotli@latest",
    "cmark@latest",
    "dash@latest",
    "delve@latest",
    "direnv@latest",
    "fish@latest",
    "flyctl@latest",
    "gh@latest",
    "git@latest",
    "gnupg@latest",
    "go@latest",
    "golangci-lint@latest",
    "graphviz@latest",
    "jq@latest",
    "k6@latest",
    "kubernetes-helm@latest",
    "nginx@latest",
    "nixpkgs-fmt@latest",
    "nodejs@20",
    "openssh@latest",
    "postgresql@latest",
    "python3@latest",
    "readline@latest",
    "ripgrep@latest",
    "sentry-cli@latest",
    "shellcheck@latest",
    "sqlfluff@latest",
    "sqlite@latest",
    "terraform@latest",
    "tree@latest",
    "vim@latest",
    "xz@latest",
    "yq@latest",
    "zsh@latest",
    "zstd@latest"
  ]
}

jeanregisser commented 8 months ago

Same here, I'm trying to use devbox global as a replacement for homebrew on macOS and I quickly hit a similar issue.

clang and things like export MACOSX_DEPLOYMENT_TARGET="10.12"; are added by eval "$(devbox global shellenv)" and breaks compiling iOS projects from the command line using xcodebuild.

Here are the errors I got:

clang-11: error: unknown argument: '-index-store-path'
clang-11: error: cannot specify -o when generating multiple output files
clang-11: warning: overriding '-mmacos-version-min=10.12' option with '-target x86_64-apple-ios11.0-simulator' [-Woverriding-t-option]

Would love to see only env variables that are strictly necessary when using devbox globally.

mjgallag commented 5 months ago

I lose colorized ls output in ohmyzsh shell due to coreutils being installed by default.

I'd prefer devbox didn't add anything to the path not explicitly listed in packages.

hlubek commented 4 months ago

We cannot build react-native projects due to this when using Devbox, which is unfortunate, because it would isolate Ruby, Node.js etc. that are dependencies for these kind of projects.

loganwishartcraig commented 4 months ago

@hlubek - We cannot build react-native projects due to this when using Devbox, which is unfortunate, because it would isolate Ruby, Node.js etc. that are dependencies for these kind of projects.

FWIW, I've had reasonable success building RN projects after tinkering w/ the xcode build config and by running a pre-build script whenever I want to build for iOS. I'm on an M1 w/ MacOS 14.1. Probably couldn't copy/paste the config below, but could be a starting point if you're interested.

Using the below for our "Bundle React Native code and images" build phase script

export SENTRY_PROPERTIES=sentry.properties
export EXTRA_PACKAGER_ARGS="--sourcemap-output $DERIVED_FILE_DIR/main.jsbundle.map"

if [[ -s "$HOME/.nvm/nvm.sh" ]]; then
  . "$HOME/.nvm/nvm.sh"
elif [[ -x "$(command -v brew)" && -s "$(brew --prefix nvm)/nvm.sh" ]]; then
  . "$(brew --prefix nvm)/nvm.sh"
fi

set -e
set -x

WITH_ENVIRONMENT="../../../node_modules/react-native/scripts/xcode/with-environment.sh"
REACT_NATIVE_XCODE="../node_modules/react-native/scripts/react-native-xcode.sh"
SENTRY_XCODE="../../../node_modules/@sentry/react-native/scripts/sentry-xcode.sh"
BUNDLE_REACT_NATIVE="/bin/sh $SENTRY_XCODE $REACT_NATIVE_XCODE"

/bin/sh -c "$WITH_ENVIRONMENT \"$BUNDLE_REACT_NATIVE\""

With these local env variables for xcode

### packages/app/ios/.xcode.env.local
export NODE_BINARY="/Users/logan/work/heylo/.devbox/virtenv/.wrappers/bin/node"
export PATH=/Users/logan/work/heylo/.devbox/virtenv/.wrappers/bin:$PATH

And this script to start the build from within a devbox shell instance

# Patch for build since devbox shadows 
# common build tools. Assumes xcode command line tools installed.
# https://github.com/jetpack-io/devbox/issues/1509
if [[ -n "$IN_NIX_SHELL" ]]; then
  CC="/usr/bin/clang"
  LD="/usr/bin/clang"
  # Strip out all nix path additions
  PATH=$(echo $PATH | tr ':' '\n' | sed '/^\/nix\/store\//d' | paste -sd ':' -)
fi

echo "Building iOS..."
PROJECT_ROOT=packages/app/ios
npx react-native run-ios --no-packager"

In this devbox config

{
  "packages": [
    "yarn@1.22",
    "nodejs@18",
    "openjdk11@latest",
    "imagemagick@7",
    "ffmpeg@5",
    "watchman@latest",
    "stripe-cli@1",
    "google-cloud-sdk@latest",
    "cocoapods@1.14.3"
  ]
}

Almost certainly a better way of doing this, but it was super painful to get even this working so I haven't continued as it's been stable.

realfresh commented 4 months ago

You'll have to forgive me, as I'm relatively new to Nix. Just trying to understand what's the cause of the issue, what's the potential fix and in the interim, what workaround can be used.

Cause

I see shellenv is generated based on function computeEnv https://github.com/jetpack-io/devbox/blob/fbeceabd544295043714f71152ebfb394633acd4/internal/devbox/devbox.go#L833

All the potentially undesired $PATH entries are coming from the output of nix print-dev-env under the key Variables.PATH.Value

The docs for nix print-dev-env say:

print shell code that can be sourced by bash to reproduce the build environment of a derivation

So I assume the reason for adding those entries into the users $PATH when running shellenv is to replicate the build environment in case invocations of the desired packages rely on specific packages/versions of dependencies in the build environment?

In which case, nothing adding those entries to the $PATH and only adding the $PATH entry to .../nix/profile/default/bin may result in certain packages not working correctly?

Solutions

Make it optional whether the nix print-dev-env paths are added to the user's shell $PATH
Allow finer-grained control over which entries get added in
Create wrappers for the package executables that modify the $PATH as needed by nix print-dev-env before they are invoked (I can see this is already done for some executables such as adbe which has a bash script with the shebang #1 /nix/store/....-bash-5.2p26/bin/bash -e and then makes $PATH modifications before invoking the command with exec -a .... Can't tell if that's something Devbox or Nix is doing)

Workarounds

After running eval "$(devbox global shellenv)", execute additional shell code to strip out the nix print-dev-env $PATH entries
Optionally, write shell code to wrap all executables inside .../nix/profile/default/bin (which are not already wrapped) with a script that makes the necessary $PATH modifications before invoking the executable

Do my current workarounds seem reasonable for the time being?

Lastly, how do you see yourself tackling this issue. I'm not sure if there are any other higher priority issues, but at least for me personally, this seems like the most troublesome.

Edit: My temporary workaround

It seems Devbox caches the output of nix print-dev-env which is stored in the file .../.devbox/.nix-print-dev-env-cache. That file contains the key Variable.PATH.Value which can be used to trim out the nix build environment paths.

So inside my .bashrc, I'm doing:

eval "$(devbox global shellenv --init-hook -r)"

TRIM_PATHS="$(jq -r '.Variables.PATH.Value' < "_the_path_to_/.devbox/.nix-print-dev-env-cache")"

export PATH=$(echo "$PATH" | tr ':' '\n' | grep -v -F -x -f <(echo "$TRIM_PATHS" | tr ':' '\n') | tr '\n' ':' | sed 's/:$//')

The end result is all the directories I don't want in my $PATH are gone, but the .devbox/nix/profile/default/bin and .devbox/virtenv/runx/bin remain.

vr8hub commented 3 months ago

Is there any movement on fixing this? I'm testing using devbox instead of brew, and this brought things to a screeching halt—the NIX coreutils version of less apparently doesn't have some of the options that the one that comes with MacOS has, and now none of my ls aliases work. devbox brought in the entire NIX coreutils to my path, along with several others, which I absolutely don't want: I installed devbox's prefixed coreutils just to prevent such a problem, only to find out that devbox is forcing them (via NIX) anyway.

I'm going to try the workaround above, but movement to fix this in devbox proper would be greatly appreciated.

hlubek commented 1 week ago

Can we have an option in Devbox to do something like stdenv = pkgs.stdenvNoCC;as mentioned in https://github.com/cachix/devenv/issues/1174 ?

savil commented 1 week ago

Devbox 0.12.0 should help with this issue:

Devbox Global has been updated to no longer set Nix environment variables or stdenv packages in your global path. This provides a cleaner environment for your Devbox packages, without interfering with the tooling installed on your host

release notes here: https://github.com/jetify-com/devbox/releases/tag/0.12.0

@hlubek does this release address your concern?

if you were asking about devbox-projects (as opposed to devbox global) behavior, then can you try devbox shell --omit-nix-env=true? --omit-nix-env is a hidden flag and thus should be considered implementation behavior (so don't rely on it necessarily) but your feedback can help understand how to make it work for Devbox projects.

hlubek commented 1 week ago

@savil Thank you so much for your reply! This actually works and I could adapt our .envrc for direnv to use this as well:

# Adapted to not set nix env for clang compatibility
eval "$(devbox shellenv --init-hook --install --no-refresh-alias --omit-nix-env=true)"

With this change the React Native build for iOS works and the system cc is used.

jetify-com / devbox