Support for using ~/.config on Mac for command line apps

[lavifb]

While not the official Apple sanctioned place for config files, the standard on the command line for MacOS tends to be using the Linux style ~/.config for config files. Can we add support for using ~/.config on Mac?

There is more discussion here: https://github.com/rust-lang-nursery/cli-wg/issues/7#issuecomment-439223000

[cjbassi]

Do cli apps on Mac also use the XDG data folder (~/.local/share/) and cache folder (~/.cache/)? If so, I'm wondering if it would be better to instead use https://github.com/whitequark/rust-xdg for command line applications at least for Base Dirs and Project Dirs, and use directories-rs for gui applications (assuming cli apps on Windows use xdg too).

[lavifb]

Yeah that definitely works for Mac but then it doesn’t for Windows. (I don’t have a windows machine but as far as I can tell XDG is not the standard there)

[lavifb]

For instance, Neovim does not use XDG on windows https://neovim.io/doc/user/starting.html#vimrc

[FranklinYu]

I want to add my answer on Stack Overflow to the reasons of using ~/.config for CLI.

If we don’t want to break compatibility, how about providing a struct CliProjectDirs?

[retep998]

assuming cli apps on Windows use xdg too

Nothing on Windows uses XDG.

[soc]

I will not support this:

Either we also change cache folders and therefore force everyone to add exclusions to macOS' backup software (Library/Caches is excluded, .cache isn't) or spread the folders into completely different places in the filesystem.

As a general advice for macOS devs: If you are developing on their platform, do what the platform owners tell you. They are the lords, you are the share-croppers.

[soc]

I closed this, but anyone should feel free to reopen if that person has new information.

[FranklinYu]

If I manually set the environment variable $XDG_CONFIG_HOME would it be respected on macOS?

[lilyball]

Cross-platform CLI apps that write to ~/Library on macOS feel wrong to me. I would really like my CLI apps to use ~/.config even on macOS. Using ~/Library/Caches is much more reasonable since, as a user, I don't have to care about that, and it won't get backed up by Time Machine. But configuration and user data should go into the appropriate CLI location.

[DanCardin]

It feels like respecting a set value for XDG_*_HOME in all cases would be ideal, and then falling back to the existing directories when unset.

While I would probably opt to not set XDG_CACHE_HOME because i don't care and thus get the existing behavior (but if someone wants to opt into a specific spot, go for it), I very much do want to set XDG_CONFIG_HOME (as much of the point of a standard location like is so that I can port my config folder to some other machine)

[FranklinYu]

It feels like respecting a set value for XDG_*_HOME in all cases would be ideal, and then falling back to the existing directories when unset.

Agree. When users set XDG_CONFIG_HOME themselves, it’s a clear indicator that this user wants configuration to be placed there. We should assume that user knows what he is doing, instead of making the decision for them.

[lilyball]

I would be very surprised if a program behaved differently with XDG_CONFIG_HOME=$HOME/.config versus XDG_CONFIG_HOME being unset.

[soc]

If I manually set the environment variable $XDG_CONFIG_HOME would it be respected on macOS?

No, it won't.

[cjbassi]

Suggestion: Add an argument to the ProjectDirs constructor called use_xdg_on_macos: bool so that users can specify if they want that feature or not. That way both GUI and CLI programs using directories-rs can work as expected since users can specify which option they want based on the type of program they are creating.

[OJFord]

As a general advice for macOS devs: If you are developing on their platform, do what the platform owners tell you. They are the lords, you are the share-croppers.

OK, so if they tell me that their $XDG_CONFIG_HOME is at /Users/lord/.config, I should use that. Except, with dirs-dev, I can't.

I am a macOS 'platform owner', as well as Linux, and my $XDG_CONFIG_HOME = $HOME/.config (i.e. explicitly, not the unset fallback per spec) on both.

I can.. understand wanting to use Library or whatever if nothing's specified, but if $XDG_* are actually present in the environment, using Library is knowingly doing what the user doesn't want.

(By the way, I found etcetera, which supports different 'strategies' for choosing base directories, such as an Xdg strategy, and the default is what we're all after - something Windowsy on Windows, and XDG on everything else. (But even if that default changes, the Xdg strategy is there and could be used even on Windows if you wanted.))

[poliorcetics]

The justification for refusing this feels very wrong to me. Apple is not all-knowing and users that are setting XDG_* variables can be expected to know what they are doing and/or use CLI tools for which they want to save the configuration files to a VCS repo.

I can understand the default case not respecting XDG_* vars since it's not recommended behaviour but a new module xdg that respects it would be nice, especially for CLI apps.

[poliorcetics]

See https://github.com/dirs-dev/dirs-rs/pull/40

[soc]

Apple is not all-knowing

Not all-knowing, but omnipotent on their hardware. They call the shots on their devices. That's the bargain you make when you buy something from them.

[OJFord]

@soc Apple does not tell you not to use environment variables to configure where data is stored on 'their' devices, that's absurd.

[soc]

@OJFord Apple defines where various kinds of data are stored. The rules do not contain an "in case you want to make up your on conventions ..." section.

[OJFord]

It defines defaults and standard practices.

Sane CLI/TUI and other programmer tools in particular respect XDG if used.

Hell, many use the $HOME or $HOME/.config without the option; respecting neither Apple nor XDG's standard.

I have absolutely no idea why you would think it makes sense togo out of your way to ignore (based on the platform) an environment variable explicitly set by the user. Is Apple paying you to uphold such ideals?

I don't currently have any macOS systems, so I don't care that much, I just think this is absurd, and I wouldn't use this crate for my own tools on that basis; not wishing it on any Mac users I may have.

[eugenesvk]

As a general advice for macOS devs: If you are developing on their platform, do what the platform owners tell you. They are the lords, you are the share-croppers I closed this, but anyone should feel free to reopen if that person has new information.

Let's try to parse the words of the macOS lord (emphasis mine)src, that might also be new information to some who can reopen :)

Where You Should Put Your App’s Files

...

• Put user data in Documents/. User data generally includes any files you might want to expose to the user—anything you might want the user to create, import, delete or edit
• Put app-created support files in the Library/Application support/ directory. In general, this directory includes files that the app uses to run but that should remain hidden from the user. This directory can also include data files, configuration files, templates and modified versions of resources loaded from the app bundle

So if a cli app manages (i.e. creates/edits/deletes them) the configs, then Application support is fine and in accordance with the express wishes of Apple.

However, if it's the common case for cli tools where the user is actually managing the files, then you're breaking your maxim by disrespecting the wishes of the platform owners and using the wrong folder!

Of course, strictly following the rules won't get you to the option of listening to the users and respecting their $XDG_CONFIG_HOME wishes, but that's still not as bad as explicitly breaking them! :)

[OJFord]

Excellent description @utkarshgupta137 👍🏻.

I only wanted to add that it just occurred to me with your CLI vs GUI description, I think when @soc writes things like:

As a general advice for macOS devs: If you are developing on their platform, do what the platform owners tell you. They are the lords, you are the share-croppers.

this is coming from the perspective that 'macOS devs' are creating apps for macOS to be used by (largely) non-developer users.

Whereas I assume anybody complaining that XDG isn't being respected is themself a developer using macOS; either writing their own tool, or using a developer tool that another 'macOS dev' created.

With that in mind, I understand the perspective a bit more - before I thought it was saying 'you are a mere user of macOS, you don't own your hardware, Apple does, do what the company says'. That said, a lot of developers use macOS, a lot of developer tools support macOS, and most developer tools respect (or almost respect, or dump in $HOME but certainly not Library/Application Support) the XDG desktop dirs spec.

I'd refine/clarify my:

I wouldn't use this crate for my own tools on that basis; not wishing it on any Mac users I may have.

that I wouldn't use it for developer tools. If I was writing a calendar app or menu bar widget or whatever, yes absolutely, Application Support does make sense, I'd have no problem using directories-rs for that sort of application.

[poliorcetics]

Even app-managed config can have a place in ~/.config, though it's true it's much rarer. git is an example where it's possible to do (almost ?) everything through the CLI but I wouldn't want its config in ~/Library/Application Support

[sdavids]

Either we also change cache folders and therefore force everyone to add exclusions to macOS' backup software (Library/Caches is excluded, .cache isn't) or spread the folders into completely different places in the filesystem.

You can address that issue by setting an extended attribute after you create the directory:

xattr -w com.apple.metadata:com_apple_backup_excludeItem \
  com.apple.backupd "${XDG_CACHE_HOME}"

xattr -w com.apple.metadata:com_apple_backup_excludeItem \
  com.apple.backupd "${XDG_RUNTIME_DIR}"

[soc]

@sdavids This library does – by design – neither create directories nor check whether they exist. Mutating extended attributes is not going to happen.

[adriangalilea]

I do understand the rationale of @soc as much as I hate the outcome.

Every time I see ~/Library/Application Support I get triggered: absurdly long names, uppercase, spaces in between that cause all sort of annoying scaping headaches...

I legit don't care if my computer crashes and burns and/or all my life's work's get's deleted as long as I can avoid using MacOS designated default dirs, I would take all the blame that came with it, but of course doing it as a Library that aims to do one thing would mean having to deal with all those user errors constantly, so I can't blame @soc for not wanting to cross that path, after all, is a MacOS flaw, and we should point our forks towards that porch. But of course nothing would change, just like it still has the worst Window Management on all the OS's available.

But while I respect the decision, and judging by the amount of petitions, why not allow for an env var that bypassed the OS detection? Wouldn't it be good for debugging too?

The only alternative would be switching to etcetera, right?

[DanCardin]

I have personally switched to etcetera over this, and i think it works perfectly well, with a nice API.

The main "problem" is, for whatever reason, new tools seem to tend to choose this lib by default, not realizing the side effect; then receive bug reports about the macos behavior afterward (like nushell). And then either manually work around it, like i've seen a few tools do, or else feel stuck because it would result in wide ranging breaking changes to switch (like nushell, who's issue has been open for 5 years).

At the same time, this library is farly clear in the readme about what it provides, and it's not in its own best interests to specifically call out this behavior any more clearly...

[soc]

Every time I see ~/Library/Application Support I get triggered: absurdly long names, uppercase, spaces in between that cause all sort of annoying scaping headaches...

Can't you create a symlink if that's your issue?

[adriangalilea]

Can't you create a symlink if that's your issue?

Nope, I share my dotfiles on mac and linux, in fact my ~ is a git repo. So no, I use XDG paths for everything I use and favor software that honors XDG envs even on mac. Went as far as to create a golang app just for being able to set XDG dirs on linux and mac and my dotfiles are also public which I use in both systems atm. I'm very happy with this setup.

[soc]

Not sure I get that – if you symlink ~./config to ~./Library/Application Support your tool wouldn't even have to care about macOS having different rules, would it?

I'd imagine the bigger issue you have is that the application naming rules are different between Linux and macOS, i. e. your application FooBar uses a foobar directory on Linux, but com.something.FooBar on macOS.

Do your tool use some kind of mapping file that translates between naming on Linux and naming on macOS?

[ssokolow]

I think the "in fact my ~ is a git repo." part is the key detail.

It sounds like they don't go with my "~/.profile_repo is a git repo and ~/.profile_repo/install.py sets up/updates symlinks" approach.

(An approach I take because it keeps git diff or git gui limited to only the files I've chosen to version without needing to use .gitignore to blacklist every single directory/file that I don't want to show up.)

[adriangalilea]

I think the "in fact my ~ is a git repo." part is the key detail.

Yup

It sounds like they don't go with my "~/.profile_repo is a git repo and ~/.profile_repo/install.py sets up/updates symlinks" approach.

Not sure I got that part

(An approach I take because it keeps git diff or git gui limited to only the files I've chosen to version without needing to use .gitignore to blacklist every single directory/file that I don't want to show up.)

You should look into stow then. I essentially .gitignore *and then whitelist things with !foo

I don't do any mapping, just all my stuff uses XDG dire. And ~ is literally a repo which I just use like you would any other repo so my config is versioned, backed up, with 0 dependenciescies, across all my unix systems. The only thing is that I need to install the software differently but I'm also automating that up. i just run 2 commands and my system is ready to go weather it be a rpi, vps, or fresh mac install. Then I git pull at the start of my .zshrc so everything is always up to date when I login to the system. Without any extra steps. And all my aliases, functions and cli tools are there.

[soc]

I don't do any mapping, just all my stuff uses XDG dire. And ~ is literally a repo which I just use like you would any other repo so my config is versioned, backed up, with 0 dependenciescies, across all my unix systems.

So your whole scheme not only requires that the applications you use are poorly ported to macOS, but also that the application authors never improve on that? 😦

[DanCardin]

So your whole scheme not only requires that the applications you use are poorly ported to macOS, but also that the application authors never improve on that? 😦

you're putting a lot of assumption and assertion about how macos CLI applications should and should not act, with (imo) very little evidence to support that things should act that way, and no other way.

Swiftlang's 1st party package manager supports XDG on macos. Also homebrew does, as an example of a piece of CLI software (originally) explicitly designed for macos. As do numerous other apps, both CLI and otherwise that have gotten feedback from enough people to decide to support it.

also https://github.com/nushell/nushell/issues/893#issuecomment-2125954485

That said, I'm pretty surprised to see this has remained open for 5 years when the solution is so simple, especially considering the resistance to changing it seems to be on ideological grounds rather than practical ones.

I'm also puzzled by those quoting the macOS Library Directory Details, which say this about Application Support:

Contains all app-specific data and support files. These are the files that your app creates and manages on behalf of the user and can include files that contain user data.

It may be a little bit subtle to those unfamiliar with macOS development, but the use of "app" is very specific here, and nushell is not an app. (If it was, it would have a bundle ID, it would be installed in /Applications, etc.)

The File System Basics document that has also been referenced says the same thing:

An app might use this directory to store a modifiable copy of resources contained initially in the app’s bundle. A game might use this directory to store new levels purchased by the user and downloaded from a server.

All content in this directory should be placed in a custom subdirectory whose name is that of your app’s bundle identifier or your company.

Again, note the references to a bundle and a bundle identifier — these are things that nushell does not have because nushell isn't an app!

and

"These are the files that your app creates and manages on behalf of the user" seems more important to me than bundle id, config files are managed directly by the user and not "on behalf of the user" ?

[soc]

Ok, that argument.

[ssokolow]

Yeah. By that reasoning, it sounds like ~./Library/Application Support/<Bundle ID> is the counterpart to Flatpak's ~/.var/app/<App ID> and nobody argues that non-Flatpak applications should put their data under ~/.var/app/.

[ssokolow]

You should look into stow then. I essentially .gitignore *and then whitelist things with !foo

Perhaps, but I generally prefer not to add new dependencies when I'm doing OK with what I have.

I don't do any mapping, just all my stuff uses XDG dire. And ~ is literally a repo which I just use like you would any other repo so my config is versioned, backed up, with 0 dependenciescies, across all my unix systems. The only thing is that I need to install the software differently but I'm also automating that up. i just run 2 commands and my system is ready to go weather it be a rpi, vps, or fresh mac install. Then I git pull at the start of my .zshrc so everything is always up to date when I login to the system. Without any extra steps. And all my aliases, functions and cli tools are there.

I use Ansible playbooks inside my repo for that. Bear in mind that my repo isn't really something I use as a roaming profile... more a template for bringing up user profiles on new machines with some dipping sauce for easing keeping it up to date, plus some Ansible playbooks for installing the system images, that gets overlaid locally with the machine-specific or sensitive bits and then persisted via backups.

(Heck, I had to restore from my playbook and a whole-$HOME nightly rsync backup a few months ago. I found it so irritating to wait for all that apt-get install-ing and flatpak install-ing and rsync-ing that I now boot via ZFSBootMenu (using the trick from their manual to hot-spare mirror /boot/efi using mdraid), have one of my tiers of backup being an hourly ZFS snapshot sync to a ZRAID mirrored pair, and am preparing to install the RMA'd NVMe drive as a ZRAID mirror for the replacement NVMe drive that I rush-ordered so I'd only have to live off my laptop for two days. With ZFS native snapshot sync, it took under 2 hours to do a from-scratch backup of my NVMe boot drive to the 7200 RPM "bulk drive" pair and it should take under 2 hours to go back in the other direction if both drives in the soon-to-be-mirrored-pair fail.)

[soc]

Yeah.

It's just that I heard variations of "here is my very specific reading of rules such that mysteriously the subset of applications I care about is exempt" ... for more than 10 years.

This is the library for the 99.8% of people who either want things to work consistently or do not want to care because all things work consistently.

It's not the library for the 0.2% of people that have to have their own special carve-out from the standard approach and subsequently ruin things for everyone else.

[ssokolow]

I find it convincing that Swift shares that interpretation of how the Apple guides repeatedly refer to the application's bundle ID.