ersiner / osx-env-sync

Synchronize OS X environment variables for command line and GUI applications from a single source
Apache License 2.0
218 stars 36 forks source link

Does not work on El Capitan #1

Open johnrizzo1 opened 8 years ago

johnrizzo1 commented 8 years ago

It looks like since many functions in launchctl were deprecated this approach does not work.

cat ~/Library/LaunchAgents/environment.plist <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

envsync local.envsync ProgramArguments sh -c launchctl setenv foo "bar" RunAtLoad

launchctl load ~/Library/LaunchAgents/environment.plist /Users/.../Library/LaunchAgents/environment.plist: Operation not permitted

launchctl setenv foo "bar" Not privileged to set domain environment.

load is apparently now a "Legacy Subcommand". The setenv subcommand also has issues.

MarkCallow commented 8 years ago

I am having problems on El Capitan too, specifically with Xcode 7. I'm not aware of other apps, that don't read my .bash_profile, that I could use to display the environment they receive so have not tested with anything else.

I do not see the errors @johnrizzo1 reports.

$ launchctl setenv DOXYGEN_BIN /Applications/Doxygen.app/Contents/Resources/doxygen
$

Perhaps this is because I have disabled SIP (System Integrity Protection). Nevertheless the environment is not being propagated.

There are no errors on the console either.

MarkCallow commented 8 years ago

I am having problems on El Capitan too, specifically with Xcode 7. I'm not aware of other apps, that don't read my .bash_profile, that I could use to display the environment they receive so have not tested with anything else.

I do not see the errors @johnrizzo1 reports.

$ launchctl setenv DOXYGEN_BIN /Applications/Doxygen.app/Contents/Resources/doxygen
$

Perhaps this is because I have disabled SIP (System Integrity Protection). Nevertheless the environment is not being propagated.

There are no errors on the console either.

MarkCallow commented 8 years ago

If I start Xcode after logging in then it sees the settings.

If Xcode is auto-started during login as part of reopening windows that were open at previous login, it does not see the settings. That was the state when I added my previous comment.

If I then exit Xcode and run it again, it sees the environment settings.

It looks like Xcode is being auto-started before the launchctl stuff is run. The behavior seems consistent so appears to be due to a deliberate ordering choice.

NOTE: I subsequently discovered a reference, that I now cannot find, that states the ordering of startup items is undefined in OS X. So it is quite possible for auto-start items to be started before the launchctl stuff that sets the environment.

ersiner commented 8 years ago

@johnrizzo1 Perhaps your user is not an "Admin" user. I cannot think of any other reasons of those errors. I'm also on El Capitan and I've never faced such error messages in my setup.

@msc- So I'm assuming your problem is solved (or never existed).

MarkCallow commented 8 years ago

The problem happens every time Xcode is auto-started during login so no the problem is not solved.

I have to make sure not to leave Xcode running when I logout or quit and restart it after login for it to see the environment.

dan-tull commented 8 years ago

I didn't see the permission errors John mentioned, but launchctl setenv is not working for me, either (I'm also using El Capitan). I was trying to get some additional entries in my PATH variable to apply inside the Atom text editor (so some plugins that invoke CLI commands find them properly).

Creating a Launch Agent plist to directly run the launchctl setenv command for the PATH will cause launchctl getenv to produce the intended path in Terminal, but not for other apps (like Atom). This is the case whether I load the service directly with launchctl load from Terminal or just log out and log back in to let the system run it.

I've jumped through the various versions of the proper recipe for setting variables durably in versions of OS X dating back to 10.5, but in 10.11 nothing I've tried has stuck. I finally had to resort to launching Atom from a Terminal.

Edited: Removed mention of environment variable expansion, which was not correct and distracts from the central issue.

ersiner commented 8 years ago

Well, this all used to be working on both Yosemite and El Capitan. Now I observe strange things:

Get the process id of Atom app running:

ps ax | grep /Applications/Atom.app/Contents/MacOS/Atom

Use launchctl procinfo command to list environment variables and other process info as controlled by OS X for the process:

sudo launchctl procinfo <process_id>

In my case the relevant output is as follows:

    inherited environment = {
        MANPATH => /usr/local/opt/erlang/lib/erlang/man:/usr/local/opt/coreutils/libexec/gnuman:/usr/share/man:/usr/local/share/man:/usr/local/opt/coreutils/libexec/gnuman
        PATH => /a:/usr/local/opt/coreutils/libexec/gnubin:/usr/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/a:/usr/local/opt/coreutils/libexec/gnubin:/usr/local/opt/go/libexec/bin:/Users/ersiner/go/bin:/Users/ersiner/bin:/usr/local/opt/go/libexec/bin:/Users/ersiner/go/bin:/Users/ersiner/bin
        LANG => en_US.UTF-8
        LC_ALL => en_US.UTF-8
        GOPATH => /Users/ersiner/go
        JAVA_HOME => /Library/Java/JavaVirtualMachines/jdk1.7.0_79.jdk/Contents/Home
        SSH_AUTH_SOCK => /private/tmp/com.apple.launchd.n4M48nI1ok/Listeners
        Apple_PubSub_Socket_Render => /private/tmp/com.apple.launchd.DZiHkV69S9/Render
    }

    default environment = {
        PATH => /usr/bin:/bin:/usr/sbin:/sbin
    }

    environment = {
        USER => ersiner
        LOGNAME => ersiner
        PATH => /usr/bin:/bin:/usr/sbin:/sbin
        SSH_AUTH_SOCK => /private/tmp/com.apple.launchd.n4M48nI1ok/Listeners
        Apple_PubSub_Socket_Render => /private/tmp/com.apple.launchd.DZiHkV69S9/Render
        HOME => /Users/ersiner
        SHELL => /usr/local/bin/bash
        __CF_USER_TEXT_ENCODING => 0x1F5:0x0:0x0
        TMPDIR => /var/folders/sd/mxnfcpyj5_lg3wd36mkyz9zc0000gn/T/
        XPC_SERVICE_NAME => com.github.atom.106592
    }

There are three sections here: inherited environment (which has all environment variables from my .bash_profile as intended), default environment (the source of mess as it seems) and environment.

When I launch the Atom console (Cmd+Alt+i) and type process.env I get (without any additional packages installed):

...
GOPATH: "/Users/ersiner/go"
HOME: "/Users/ersiner"
JAVA_HOME: "/Library/Java/JavaVirtualMachines/jdk1.7.0_79.jdk/Contents/Home"
LANG: "en_US.UTF-8"
LC_ALL: "en_US.UTF-8"
LOGNAME: "ersiner"
MANPATH: "/usr/local/opt/erlang/lib/erlang/man:/usr/local/opt/coreutils/libexec/gnuman:/usr/share/man:/usr/local/share/man:/usr/local/opt/erlang/lib/erlang/man:/usr/local/opt/coreutils/libexec/gnuman"
NODE_ENV: "production"
NODE_PATH: "/Applications/Atom.app/Contents/Resources/app.asar/exports"
PATH: "/usr/bin:/bin:/usr/sbin:/sbin"
SHELL: "/usr/local/bin/bash"
SSH_AUTH_SOCK: "/private/tmp/com.apple.launchd.n4M48nI1ok/Listeners"
TMPDIR: "/var/folders/sd/mxnfcpyj5_lg3wd36mkyz9zc0000gn/T/"
USER: "ersiner"
...

So the PATH environment variable is being overrided by the "default environment" when launching the process. Other variables like GOPATH or JAVA_HOME have intended values in the Atom environment.

I could not find any reference to these different "environments" in Apple's documentation. Maybe there have been changes to how these all handled in the system recently. I'll dig more soon but you may want to share your our outputs of the same commands here which may help. Also outputs from Yosemite would be great if any of you still using it.

dan-tull commented 8 years ago

I do indeed also have a default environment entry that is stomping PATH (as reflected by process.env in the console in Atom).

My inherited environment is shorter (I had only set PATH via launchctl) but I do also have SSH_AUTH_SOCK and Apple_PubSub_Socket_Render, which are apparently being set by the system.

It looks like we're hitting the same issue. Good find with that procinfo subcommand. I look forward to hearing if you find out more about the default environment that's hosing this up.

dan-tull commented 8 years ago

Curious. When I launch Atom from Terminal (via Atom's own atom command line shortcut, available if you're run "Install Shell Commands" in the app menu), the inherited, default, and environment sections remain but the default one does not appear to take precedence and the augmented PATH from inherited comes through unmolested.

I notice there's an extra argument to the program: --path-environment=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

I'd have to go digging in Atom's repo, but it looks like they've run into this and added an extra command line argument to funnel through and re-stomp the PATH environment variable.

Update: Confirmed: https://github.com/atom/electron/issues/550

MarkCallow commented 8 years ago

I suffered an indentical problem to @dan-tull's in Yosemite (could never modify $PATH) but it was with Xcode. Eventually I gave up and found workarounds that did not require modifying $PATH. The problem continues in El Capitan. It might be that whatever launchctl behavior was applied to Xcode in Yosemite is now applied to all applications.

The problem with auto-start that I reported earlier in this thread is for any environment variable other than $PATH.

mgol commented 8 years ago

@ersiner I confirm your observations: launchctl procinfo shows correct PATH in the inherited environment section but the default environment one is the one being used by the app.

xapple commented 8 years ago

Xcode 7.0+ doesn't evaluate environment variables by default. The old behaviour can be enabled with this command:

defaults write com.apple.dt.Xcode UseSanitizedBuildSystemEnvironment -bool NO
dan-tull commented 8 years ago

I'm presuming the "improved environment sanitization" security fixes are behind this: http://www.securityweek.com/apple-releases-patch-os-x-vulnerability-exploited-wild

Inconvenient, but not without cause from a security standpoint, I guess. So far, I haven't found any details on how this sanitization works or how to alter the default PATH value.

mgol commented 8 years ago

@xapple

Xcode 7.0+ doesn't evaluate environment variables by default. The old behaviour can be enabled with this command:

defaults write com.apple.dt.Xcode UseSanitizedBuildSystemEnvironment -bool NO

This didn't help; SourceTree still sees PATH only as /usr/bin:/bin:/usr/sbin:/sbin.

I know it's possible to fix as it worked for me on my old MacBook before I changed it about 2 weeks ago. But I don't know how. :/

MarkCallow commented 8 years ago

With the latest El Capitan (10.11.5) and Xcode (7.3.1) none of my environment variables are visible in Xcode even when the auto-start problem is avoided. I can't say which update caused the new breakage as I had not needed any of my environment in Xcode for a while. Unfortunately @xapple's

defaults write com.apple.dt.Xcode UseSanitizedBuildSystemEnvironment -bool NO

didn't help me either. The before and after "environment" displayed by

sudo launchctl procinfo <process_id>

is identical and does not include any of my environment variables from "inherited environment".

@dan-tull, @mgol, even prior to Xcode 7 & El Capitan, osx-env-sync failed to modify $PATH as seen by Xcode. I gave up on that a long time ago*. It did though make other environment variables visible to Xcode.

* Since I only need the modified $PATH in an Action build-phase script, my workaround is to set up the action to use a login shell so that the shell reads my environment. The default is /bin/sh which does not read any startup files. I use /bin/bash -l instead.

MarkCallow commented 8 years ago

Experimentation reveals that

defaults write com.apple.dt.Xcode UseSanitizedBuildSystemEnvironment -bool NO

only has an effect when launching Xcode.app from the command line. In that case it indeed allows Xcode to inherit the environment, including $PATH. When launching Xcode from launchpad, it has no effect. Why does Apple make this so damn difficult? BTW, xcodebuild defaults to UseSanitizedBuildSystemEnvironment=NO.

Important Note for anyone else wrestling with this: xcodebuild -showBuildSettings only shows official settings and their values. It does not show general environment variables. E.g, it will show PATH and its value but will not show some random environment variable, e.g, one I am trying to use MOLTENVK_HOME, even when it is set in xcodebuild's environment. So it can't be used for testing receipt of environment variables from launchctl.

MarkCallow commented 8 years ago

I just filed bug #26690137, "Xcode launched from launchpad does not inherit environment", on Apple's bug reporter.

mukhark commented 8 years ago

I was having this same problem with Eclipse. My environment is Mac OSX El Capitan 10.11.3, Eclipse Luna 4.4.2, build 20150219-0600.

I was running a maven build inside Eclipse which was using the maven-exec plugin to execute another program. The path variable used by Eclipse (when launched from the dock) was "/usr/bin:/bin:/usr/sbin:/sbin" from the default environment, and so my build was failing because the executable that maven was trying to call is not on that path. The PATH var in my shell was set so that the program was found.

Looking at the man page for launchctl, I saw that one of its other subcommands is "config" which the man page says can be used to set "the PATH environment variable for all services within the target domain."

So I ran the following command in a terminal window:

sudo launchctl config user path $PATH

The drawback is that this requires a reboot of the machine. But once my machine was rebooted, and I relaunched eclipse, my eclipse app was using the same path as I had set in the terminal window. When I run launchctl procinfo it still shows the same default environment, but the environment uses the PATH that I set and not the PATH from default.

optimaict commented 8 years ago

@mukhark thank you!.. it solved my $PATH issue in Launchd while configuring Odoo ERP to load at boot on El Capitan 10.11.5.... the advantage of your solution is that its a persistent/permanent fix

MarkCallow commented 7 years ago

With Xcode 8 + OS X 10.11.6 Xcode's environment is set to the inherited environment (including $PATH) when launched from launchpad or the command line, presumably provided UseSanitizedBuildSystemEnvironment is set to NO; I haven't tried with it set to YES.

In other words bug #26690137 that I reported to Apple has been fixed, though they have not closed the bug.

However when Xcode is auto-started during login it still fails to receive the inherited environment.

MarkCallow commented 7 years ago

I tried to use a variable from my environment in Xcode 8 today. I found that it is not receiving the inherited environment when started from launchpad. I do not know why it worked 5 days ago. When I checked it then, and it appeared to be working, I tried several more times and each time Xcode inherited the environment. There have been no updates to either Xcode or OS X in the intervening period. The only change is that I have rebooted several times since. Sighhh!

FranklinYu commented 7 years ago

@msc- You mentioned

However when Xcode is auto-started during login it still fails to receive the inherited environment.

This depends on how you "auto-start" Xcode. Applications started by "Preferences -> Users & Groups -> Login Items" (shown below) are started before the user launch agent read the property lists.

Login Items

JasonCoombs commented 2 years ago

Got here from https://dani.gg/en/467-add-usr-local-bin-to-system-path-variable-on-mac-os/

using /bin/zsh on MacOS:

% set
...
PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin::/opt/X11/bin:/Library/Apple/usr/bin
path=( /usr/local/bin /usr/bin /bin /usr/sbin /sbin /opt/X11/bin /Library/Apple/usr/bin )

% env
...
PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Library/Apple/usr/bin

% which diff
/usr/bin/diff

% ls -l /usr/local/bin/diff
lrwxr-xr-x  1 jasoncoombs.ceo  admin  32 Feb 22 16:09 /usr/local/bin/diff -> ../Cellar/diffutils/3.8/bin/diff

% /usr/local/bin/diff --version
diff (GNU diffutils) 3.8
Copyright (C) 2021 Free Software Foundation, Inc.

with GNU diffutils installed via brew.

I'm trying to figure out why the priority position of /usr/local/bin is being ignored in my path.

Executing programs found only in /usr/local/bin works as expected, suggesting something is overriding the path order.

SOLVED: https://apple.stackexchange.com/questions/226149/path-not-being-honored#comment361591_286561

% unhash diff
% which diff
/usr/local/bin/diff

(rebooting or logging off would perhaps also have solved this problem. path was not being checked because of hash table.)