Open starcraft66 opened 4 years ago
I should add that applications are already symlinked into ~/.nix-profile/Applications
by nix-env(?) but this is not enough for them to be picked up by Spotlight. nix-darwin
goes a step further by linking these apps into ~/Applications
where Spotlight does search for apps.
https://github.com/LnL7/nix-darwin/blob/master/modules/system/applications.nix
Here's what I added to my configuration to make this work
EDIT : the example has been further simplified
{
# Install MacOS applications to the user environment if the targetPlatform is Darwin
home.file."Applications/home-manager".source = let
apps = pkgs.buildEnv {
name = "home-manager-applications";
paths = config.home.packages;
pathsToLink = "/Applications";
};
in mkIf pkgs.stdenv.targetPlatform.isDarwin "${apps}/Applications";
}
This task is now handled in my PR #1460
I'm seeing a conflict between the above code and nix-darwin
:
Creating home file links in /Users/johnw
ln: failed to create symbolic link '/Users/johnw/Applications/Home Manager Apps': Permission denied
This is because ~/Applications
is a symlink into my Nix store that cannot be modified, while it looks like home-manager
is trying to make a new directory directly within ~/Applications
.
You are right. The logic in nix-darwin to symlink applications to ~/Applications
is here.
if [ ! -e ~/Applications -o -L ~/Applications ]; then
ln -sfn ${cfg.build.applications}/Applications ~/Applications
elif [ ! -e ~/Applications/Nix\ Apps -o -L ~/Applications/Nix\ Apps ]; then
ln -sfn ${cfg.build.applications}/Applications ~/Applications/Nix\ Apps
else
echo "warning: ~/Applications and ~/Applications/Nix Apps are directories, skipping App linking..." >&2
fi
AFAIK there are many applications that create folders within ~/Applications
so nix-darwin
's logic seems wrong.
The first if
should be removed altogether.
Maybe @lnl7 could provide more input?
@jwiegley Is it nix-darwin that takes ~/Applications
or something else? I don't know macOS so I assumed that ~/Applications
is a pre-existing directory for all users. I guess that assumption is wrong? Should I revert the d3aee544b686a72b2bb7eeb379f72c6b6b2665b7 commit?
On a brand new Mac ~/Applications already existed in my case
I've disabled the feature in the above commit until we've come to an agreement with nix-darwin 🙂
I don't see a reason why this should also be disabled for Darwin users who don't use nix-darwin, could you make it an option instead?
I might have something unusual about my setup, but I've found that symlinks don't get picked up by spotlight, so symlinking the app dir (or the individual apps) don't cause them to show up in my spotlight search (eg cmd-space + "emacs" only shows web results).
After some debugging, I found that spotlight ignores symlinks but will index aliases. Aliases are kind of awful to work with - you can't seem to create them from the CLI without pinging finder via AppleScript / osascript.
I've added the following to my config which seems to make things work:
home.activation = {
aliasApplications = lib.hm.dag.entryAfter [ "writeBoundary" ] ''
app_folder=$(echo ~/Applications);
for app in $(find "$genProfilePath/home-path/Applications" -type l); do
$DRY_RUN_CMD rm -f $app_folder/$(basename $app)
$DRY_RUN_CMD osascript -e "tell app \"Finder\"" -e "make new alias file at POSIX file \"$app_folder\" to POSIX file \"$app\"" -e "set name of result to \"$(basename $app)\"" -e "end tell"
done
'';
};
(in the context of my dot files: https://github.com/nuance/dotfiles/blob/master/nix/environments/macos.nix#L11-L19)
Happy to make a PR for this or let someone else use this code if it looks reasonable.
An alternative solution I have (sort of) working locally is to cp -r --symlink
the .Apps or maybe even just symlink the Contents/
subdir but at least one app (Anki) has issues with that approach.
I don't have a Mac any more, but isn't it sufficient to symlink from ~/Applications/Nix Apps
to ~/.nix-profile/Applications
?
It is not. As @nuance mentioned, Finder ignores symlink for whatever stupid reason.
Although you could try aliasing the whole directory instead of the indiviual apps perhaps.
For what it’s worth, wasn’t able to figure out how to alias a directory with applescript, but I’m barely proficient at it so it’s possible I’m missing something simple. Aliasing the folder definitely seems a little cleaner and deals with removing applications, which my solution currently does not (the alias will point to whatever previous install exists, which might cause interesting behavior in the event nix GCs it...).
On Tue, Jan 19, 2021 at 1:58 PM Atemu notifications@github.com wrote:
It is not. As @nuance https://github.com/nuance mentioned, Finder ignores symlink for whatever stupid reason.
Although you could try aliasing the whole directory instead of the indiviual apps perhaps.
— You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/nix-community/home-manager/issues/1341#issuecomment-763053127, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAACBDAXV4SEIBOL5CGJ2L3S2XI3VANCNFSM4OCG3S7A .
I'm guessing that people who're having problems with Spotlight installed Nix using the post-Catalina installer. The newer installer creates the Nix store on a separate filesystem, which prevents files there from being indexed by Spotlight.
There are 2 known solutions for this:
However, there are problems with both one way or another. The same issue came up on Homebrew numerous times, and they never found a satisfactory answer.
With the symlink approach, things should work just as it did before. However, I haven't upgraded from Mojave, so I can't be 100% sure whether it would work on Catalina and later. But the problem is, Spotlight will pick up all applications from the Nix store and not only the ones in the current profile. This means it's very difficult to tell which one to choose when you have multiple profile generations in your Nix store:
With the alias approach, you won't have the same problem as long as the Nix store is out of reach of Spotlight. However, the alias won't be categorized under "Applications" and will get pushed far down the list:
Comparing the two, the alias approach slightly looks more usable if it turns out to work reliably. There are past reports of the "right click → Open With" functionality not working when using the alias approach, but I haven't encountered that problem so far so it may have been resolved.
Looking at Homebrew's past struggles with aliases, the main issue seemed to be about keeping aliases in a healthy state between upgrades. Home Manager might not suffer from this problem because it can delete the Home Manager Apps
directory and recreate the aliases from scratch on each upgrade. I don't know how reliable this will be when the aliases cross filesystem boundaries though.
FWIW, I've been using the cp -rs
approach for Emacs successfully and maybe it's just Anki for which it doesn't work (I don't manage any other apps with Nix).
I've got an experimental branch here but it might not work in some cases because --no-preserve=mode
strips exec permission too. I wanted to find a way to only reset the write permission but you could also just chmod +w
afterwards I guess.
If it turns out that doesn't work with many other apps, we could also resort to just copying the .apps. (hard- or reflinks would be cross-device unfortunately.)
I've just tried the cp -rs
approach, but it appears spotlight won't index those too. The apps would appear on Launchpad and you can launch them alright, but it won't appear on spotlight searches.
Here are the steps I followed to check:
nix run nixpkgs.coreutils -c cp -rs /Volumes/MacVim/MacVim.app ~/Applications
MacVim.app
is symlinked from the DMG image to simulate the post-Catalina situation in which the Nix store resides on a separate filesystem.
Copying the apps could be tricky as well because if I understand correctly, apps built in nixpkgs have no guarantee of being relocatable.
Yeah doing it with MacVim doesn't work for me either. Weird that it does work for Emacs though.
Currently I copy the applications, and it's working for me so far. cp -fHRL
resolves all symlinks, and though the copied apps may have references to files in the nix store, those won't be garbage collected as long as the "original" applications are still in the current profile.
home.activation = {
copyApplications = let
apps = pkgs.buildEnv {
name = "home-manager-applications";
paths = config.home.packages;
pathsToLink = "/Applications";
};
in lib.hm.dag.entryAfter [ "writeBoundary" ] ''
baseDir="$HOME/Applications/Home Manager Apps"
if [ -d "$baseDir" ]; then
rm -rf "$baseDir"
fi
mkdir -p "$baseDir"
for appFile in ${apps}/Applications/*; do
target="$baseDir/$(basename "$appFile")"
$DRY_RUN_CMD cp ''${VERBOSE_ARG:+-v} -fHRL "$appFile" "$baseDir"
$DRY_RUN_CMD chmod ''${VERBOSE_ARG:+-v} -R +w "$target"
done
'';
};
I think if home manager were to change to handle this use case, it might add an option for copying targets rather than linking them, but how could it do this safely? It would need to be able to overwrite targets that already exist.
Like this?
https://github.com/Atemu/home-manager/blob/darwin-copy-apps-fully-wip/modules/targets/darwin.nix
I'm not sure how well reverting back to a symlink could work but atm HM just says the dir is in the way of the symlink and needs to be removed first which I think is fine.
Thank you for your contribution! I marked this issue as stale due to inactivity. If this remains inactive for another 7 days, I will close this issue. Please read the relevant sections below before commenting.
* If this is resolved, please consider closing it so that the maintainers know not to focus on this. * If this might still be an issue, but you are not interested in promoting its resolution, please consider closing it while encouraging others to take over and reopen an issue if they care enough. * If you know how to solve the issue, please consider submitting a Pull Request that addresses this issue.
* If you are also experiencing this issue, please add details of your situation to help with the debugging process. * If you know how to solve the issue, please consider submitting a Pull Request that addresses this issue.
If you have nothing of substance to add, please refrain from commenting and allow the bot close the issue. Also, don't be afraid to manually close an issue, even if it holds valuable information.
Closed issues stay in the system for people to search, read, cross-reference, or even reopen--nothing is lost! Closing obsolete issues is an important way to help maintainers focus their time and effort.
Still relevant.
I am on Big Sur and launchpad is behaving very weird, looks like it copies or caches the apps in ~/Applications/Home\ Manager\ Apps
i.e. if an app inside that folder is updated (it now links to a different package in nix store let's say), launchpad still points to the original app in nix store. Maybe the other approach of copying applications instead of symlinking them doesn't have this problem ?
Has anyone submitted a feedback report to Apple for the general issue of symlinks not being followed during Spotlight indexing?
Come to think of it, do symlinks work from indexable filesystem to another indexable filesystem?
What I mean is:
$ touch foo
$ ln -s foo bar
and then trying to search for bar
.
If this works, then the issue is that files / directories on a non-indexable filesystem aren't indexed when symlinked to a path on an indexable filesystem, while if it doesn't, the issue is that symlinks aren't indexed at all by Spotlight.
Has anyone submitted a feedback report to Apple for the general issue of symlinks not being followed during Spotlight indexing? @winterqt
This is a well known issue that the homebrew folks have known for over 10 years. (This means they likely have reported the issue to Apple)
https://apple.stackexchange.com/q/23653 https://github.com/Homebrew/legacy-homebrew/issues/8699
This is a well known issue that the homebrew folks have known for over 10 years.
(This means they likely have reported the issue to Apple)
@berbiche Ah, I see. That's... annoying. 😕 Thank you for the information, though!
Currently I copy the applications, and it's working for me so far.
cp -fHRL
resolves all symlinks, and though the copied apps may have references to files in the nix store, those won't be garbage collected as long as the "original" applications are still in the current profile.home.activation = { copyApplications = let apps = pkgs.buildEnv { name = "home-manager-applications"; paths = config.home.packages; pathsToLink = "/Applications"; }; in lib.hm.dag.entryAfter [ "writeBoundary" ] '' baseDir="$HOME/Applications/Home Manager Apps" if [ -d "$baseDir" ]; then rm -rf "$baseDir" fi mkdir -p "$baseDir" for appFile in ${apps}/Applications/*; do target="$baseDir/$(basename "$appFile")" $DRY_RUN_CMD cp ''${VERBOSE_ARG:+-v} -fHRL "$appFile" "$baseDir" $DRY_RUN_CMD chmod ''${VERBOSE_ARG:+-v} -R +w "$target" done ''; };
I think if home manager were to change to handle this use case, it might add an option for copying targets rather than linking them, but how could it do this safely? It would need to be able to overwrite targets that already exist.
I wonder if an rsync
(or similar) based solution for copying the files would be more or less efficient at copying large Applications
directories.
We'd have the advantage of not copying (large) files needlessly, which may be better in the end?
I don't think rsync
would work particularly well here because it can't rely on timestamps since they're all 1970-01-01.
Ah, I did not realize that, apologies.
Are the files in the Nix store like that, or is it the symlinks, or...? edit: files in the Nix store always have timestamps of 1970-01-01
Finder looks at the info.plist file, the rest can be symlinks. So if you make applications/Foo.app a dir and under that you symlink everything, I think it will work.
I don't have a Mac any more to test
This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/mac-applications-installed-by-nix-are-not-loaded-by-spotlight/14129/1
This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/mac-applications-installed-by-nix-are-not-loaded-by-spotlight/14129/2
I’ve been using rsync
in an activation script to populate ~/Applications
(with --checksum
because the timestamps never change). I tried updating it to copy just Info.plist
per @wmertens, but I ran into problems with Visual Studio Code. The app would start up, but it would show just a blank window. It’s the only Electron-based app I use with home-manager, so I don’t know if others have problems.
Looking at the output in the terminal, I saw it wasn’t able to find its helper apps. I was able to work around the issue by copying over Contents/Frameworks
and Contents/MacOS
. This is what I’ve got currently. Aside from the Electon issue, I also tried to avoid excessive nagging by macOS when I switched generations by linking to the original location in the store. I’m not sure whether it’s necessary to reset the timestamps, but I like that they match the store.
I saw it wasn't able to find its helper apps
I assume the error you were getting was "Unable to find helper app?" If so, that's thrown here:
This uses a function called base::PathExists
from Chromium, which is implemented here:
(I'm assuming they're using the POSIX implementation on macOS, as I couldn't find a macOS specific implementation of this function)
The access
function used here is documented as:
access() checks whether the calling process can access the file pathname. If pathname is a symbolic link, it is dereferenced.
So... it shouldn't be failing, at least there? Unless the macOS implementation is different?
@reckenrode Do you mind running VS Code with dtruss
while symlinking everything except Info.plist
, so we can see exactly what it's doing?
I assume the error you were getting was "Unable to find helper app?" If so, that's thrown here:
That’s the one. It prints the following in the terminal:
[1481:0719/100344.673048:FATAL:electron_main_delegate_mac.mm(71)] Unable to find helper app
[0719/100344.811427:WARNING:crash_report_exception_handler.cc(240)] UniversalExceptionRaise: (os/kern) failure (5)
@reckenrode Do you mind running VS Code with
dtruss
while symlinking everything exceptInfo.plist
, so we can see exactly what it's doing?
Not at all. I disabled SIP, ran dtruss -asl -W Electron
as root, then ran VS Code as my normal user. I uploaded the logs here. There are three.
trace-workaround.log
— VS Code working as expected;trace-symlinks.log
— VS Code with symlinks (failing); andtrace-children.log
— same as above except -f
was also used with dtruss
.Thank you!
When using symlinks, right before the "Unable to find helper app" error is thrown, the access
syscall is used, as I expected:
1112/0x2857: 48498 12 10 access("/nix/store/zhahs54xcls7fhw4vc8bpm2dsl5drm3w-vscode-1.57.1/Applications/Visual Studio Code.app\0", 0x4, 0x0) = -1 Err#1
<some other stuff...>
1112/0x2857: 48514 3 1 access("/nix/store/zhahs54xcls7fhw4vc8bpm2dsl5drm3w-vscode-1.57.1/Applications/Visual Studio Code.app/Contents/Frameworks/ Helper.app/Contents/MacOS/ Helper\0", 0x0, 0x0) = -1 Err#2
So, it's correctly dereferencing symlinks.
Here's something strange I noticed while typing this up: notice the space in between the last /
and the word Helper
? If you go into the Electron shell main app delegate, you'll see this:
We can conclude based on this that the name
string is blank. name
is derived from one of two things: either the ELECTRON_APP_NAME
definition, and if that doesn't exist, the result of GetApplicationName()
.
On macOS, this function retrieves a value from the application info dictionary, using a magic constant that I can't find the value of for the life of me. (see https://github.com/electron/electron/blob/44491b023ac2653538b7ac36bb73f3085a938666/shell/common/application_info_mac.mm#L29)
In conclusion, I'm stumped.
What's the output of ls /nix/store/zhahs54xcls7fhw4vc8bpm2dsl5drm3w-vscode-1.57.1/Applications/Visual Studio Code.app/Contents/Frameworks
? I want to see what the actual helpers that it should be finding are called.
I can't find the value of for the life of me.
Are you referring to kCFBundleNameKey
? That’s a framework constant containing the human-readable name of the bundle. It’s defined here.
This Stack Overflow post indicates it can return null in certain circumstances.
Are you referring to kCFBundleNameKey?
I was, yes. Thank you, my apologies for not thinking of searching for it elsewhere. 😅
When copying the frameworks, this value is seemingly able to be found:
2083/0x44ea: 25725 18 15 access("/Users/reckenrode/Applications/Home Manager Apps/Visual Studio Code.app/Contents/Frameworks/Code Helper (Renderer).app\0", 0x4, 0x0) = 0 0
<some other stuff...>
2083/0x44ea: 32993 5 4 access("/Users/reckenrode/Applications/Home Manager Apps/Visual Studio Code.app/Contents/Frameworks/Code Helper (Renderer).app/Contents/MacOS/Code Helper (Renderer)\0", 0x0, 0x0) = 0 0
I really don't see why this wouldn't be working with symlinks, as they're dereferenced correctly with the calls to access
...
In conclusion, I'm stumped.
I noticed the same thing. After some digging, it appears to fail when the bundle’s executable is symlinked. NSBundle.mainBundle
returns nil
, which causes NSDictionary.objectForKey:
to return nil
, and SysNSStringToUTF8
returns an empty string when it gets nil
. That’s how the app name is blank.
I was able to reproduce this behavior with a simple app that just displays the application’s name from its bundle. When I symlink Contents/MacOS
, it’s blank. When I copy it, it displays correctly.
I tried setting up the helper apps like the main VS Code app (with copied Info.plist
and MacOS
), but I just got a different error this time.
[0719/114351.219673:WARNING:process_memory_mac.cc(93)] mach_vm_read(0x7ffee2b9c000, 0x2000): (os/kern) invalid address (1)
[0719/114351.363508:WARNING:crash_report_exception_handler.cc(240)] UniversalExceptionRaise: (os/kern) failure (5)
What's the output of
ls /nix/store/zhahs54xcls7fhw4vc8bpm2dsl5drm3w-vscode-1.57.1/Applications/Visual Studio Code.app/Contents/Frameworks
? I want to see what the actual helpers that it should be finding are called.
> ls '/nix/store/zhahs54xcls7fhw4vc8bpm2dsl5drm3w-vscode-1.57.1/Applications/Visual Studio Code.app/Contents/Frameworks'
'Code Helper (GPU).app' 'Electron Framework.framework' Squirrel.framework
'Code Helper (Renderer).app' Mantle.framework
'Code Helper.app' ReactiveObjC.framework
After some digging, it appears to fail when the bundle’s executable is symlinked.
NSBundle.mainBundle
returnsnil
, which causesNSDictionary.objectForKey:
to returnnil
, andSysNSStringToUTF8
returns an empty string when it getsnil
. That’s how the app name is blank.
Interesting, thanks.
I wonder if the underlying NSBundle.mainBundle
issue is worth reporting to Apple? Also, I'm curious if there's a different mechanism that Electron could adopt for finding the helper apps that doesn't use NSBundle.mainBundle
, that would work just as well as the current one?
I wonder if the underlying
NSBundle.mainBundle
issue is worth reporting to Apple?
I submitted FB9362222 via Feedback Assistant. I linked this issue and included the test app to reproduce. I also asked in the feedback if this is intended behavior, which it may be.
Also, I'm curious if there's a different mechanism that Electron could adopt for finding the helper apps that doesn't use
NSBundle.mainBundle
, that would work just as well as the current one?
According to the application distribution tutorial, setting your product name should set the value of ELECTRON_APPLICATION_NAME
, which would allow VS Code to find its helpers. Not sure why VS Code doesn’t. I assume its build process just sets the names of the contents of the bundle per the section on rebranding.
However, I’m concerned that anything using bundle APIs could have problems.
tries to run NetNewsWire
Welp. Even with the workaround, NetNewsWire crashes without being able to find its main storyboard. I have to copy Resources
into the bundle to get it to launch.
Welp. Even with the workaround, NetNewsWire crashes without being able to find its main storyboard. I have to copy
Resources
into the bundle to get it to launch.
Here's something that probably won't work, but I guess is worth giving a try: I'm not sure how you'd efficiently do this in a shell script, but can you create the actual Resources
directory, and symlink the files into it?
An example of the tree I'm thinking of:
NetNewsWire.app/ (directory)
Resources/ (directory)
SomeFolder/ (directory)
SomeFile (symlink)
Main.storyboard (symlink)
Again not sure if this will do anything, but it's worth a shot?
That’s basically cp -rs
. I tried that, and it still crashes:
021-07-19 13:26:12.489 NetNewsWire[7544:140906] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: 'There doesn't seem to be a valid compiled storyboard at path '/Users/reckenrode/Applications/Home Manager Apps/NetNewsWire.app/Contents/Resources/Base.lproj/Main.storyboardc'. Info.plist exists, but is invalid. Contents:
(null)'
I then tried copying the Info.plist
in that storyboard, and it crashed with a different message.
2021-07-19 13:28:37.389 NetNewsWire[7618:143429] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[__NSDictionaryM setObject:forKey:]: object cannot be nil (key: MainMenu)'
So just to be clear, if you create the Foo.app/Contents and then symlink all the items under it, it doesn't work?
It sometimes works. kitty works. Secretive works (more or less). But it sometimes doesn’t. Neither Visual Studio Code nor NetNewsWire work. VS Code fails because it can’t use the NSBundle
APIs to read its Info.plist
. I assume something similar is happening with NetNewsWire (except with its storyboards).
Edit: This is on macOS 11.4. It’s possible that symlinking works on older versions. I don’t have any pre-11 systems to test.
Secretive works (more or less). But it sometimes doesn’t.
Can you describe this in a bit more detail? I know the maintainer of Secretive is very diligent with regards to fixing bugs, so they may be able to issue a fix, if it's something Secretive can fix. (I see the same with NetNewsWire after a cursory glance at their issue tracker, but that's a much larger codebase.)
I don't know what the best way to fix these would be, whether it's Apple fixing / changing the behavior of the API, or getting upstream fixes, or what, to be honest.
Can you describe this in a bit more detail? I know the maintainer of Secretive is very diligent with regards to fixing bugs, so they may be able to issue a fix, if it's something Secretive can fix. (I see the same with NetNewsWire after a cursory glance at their issue tracker, but that's a much larger codebase.)
Secretive opens, but I can’t interact with it until I hit ⌘Q. Otherwise, everything seems there and functional.
I don't know what the best way to fix these would be, whether it's Apple fixing / changing the behavior of the API, or getting upstream fixes, or what, to be honest.
I ended up reverting my activation script to rsync
†the apps into ~/Applications/Home Manager Apps
because it just works (at the cost of disk space). I like the idea of symlinking, but it seems brittle without knowing exactly what requirements Apple’s APIs impose. I’m also skeptical about convincing upstream projects (especially non-free ones) to make changes just to accommodate this case.
†rsync --archive --checksum --chmod=-w --copy-unsafe-links --delete
to be specific.
This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/mac-applications-installed-by-nix-are-not-loaded-by-spotlight/14129/7
â€
rsync --archive --checksum --chmod=-w --copy-unsafe-links --delete to be specific.
@reckenrode Curious: how efficient is this approach? For example, how long does it make an activation with no changes to ~/Applications
? (as it still has to hash everything)
Issue description
I noticed that when I install an application on macOS that includes a GUI launcher (emacs in my case) using nix-darwin,
Emacs.app
is symlinked into~/Applications/Nix Apps
so that things like Spotlight can find and launch Emacs without requiring me to have a dedicated terminal open to start emacs.I think home-manager should adopt this behaviour and symlink its apps into
~/Applications/Home Manager Apps
or something like that to greatly improve the experience of launching GUI apps on macOS/darwin.Meta
Not sure what other useful information to add, this is more of a feature request than a bug.