Expose XDG conventions on macOS

[xvxx]

Hi there, and thank you for this crate.

macOS is in the weird position of having its own conventions for Mac apps, but also being a UNIX system. A lot of programs developers install aren’t exclusively Mac programs, but also work on Linux, BSDs, etc.

For example, the fish shell, mpv, spotify-tui, GitHub’s “hub”, and the kitty terminal are a few I use personally on Linux and Mac. On both OSes they all put their config files in the same place: ~/.config.

Meanwhile, the ~/Library/Application Support directory on my Mac seems to be exclusively used by native Mac applications. Applications that I would launch through the GUI, and which aren’t in my $PATH.

It would be great if this crate made the XDG conventions available on Macs too, so developers using this crate have the option of which to use. I have a silly little Gopher client, phetch, that runs on Linux and Mac. It uses ~/.config/phetch, which plays nicely with the tools I listed above and is convenient for version control, discovery, etc. I’m not able to use this crate, because on Mac the config dir would be ~/Library/Application Support/phetch.

I’ve also run into this while contributing to the Blightmud MUD client, which initially kept its config in ~/Library/Application Support/blightmud. In that case, your config files are mostly Lua scripts - code that you want to add to or change regularly - and the application isn’t specific to macOS. We eventually switched to ~/.config/blightmud so it’s the same on *nix and Mac, but we couldn’t find a way to make that change while continuing to use dirs-next.

If the option of which to use were made available to developers, I imagine the thought process would go something like: If you’re building a native Mac application that’ll live in /Applications and has a GUI, you use the Apple guidelines (like ~/Library/Application Support). However, if you’re building a UNIX application that isn’t specific to Mac, you could use the XDG conventions.

While my request may seem to go against the Standard Directories guidelines that Apple provides, I think it’s actually true to the guidelines because of the first entry:

/Applications    This directory is where you install apps intended for use by all users of a computer

In other words, I believe that the guidelines are for native Mac applications that live in /Applications. Random UNIX utilities like ytop and bandwhich and mpv don’t need to follow these guidelines, and in fact it adds a bit of friction when they do. It would be great if they could also make use of this crate.

Thank you!

[nirvdrum]

@xvxx You probably already know this, but following the XDG convention would put the config in $XDG_CONFIG_HOME, which happens to default to $HOME/.config. I.e., hard-coding to ~/.config is wrong on Linux if an application is adhering to the XDG specification. I only mention that because you could still have diverging path locations given a Mac user almost certainly doesn't have XDG_CONFIG_HOME set while a Linux user may.

[xvxx]

@nirvdrum I don’t mean to imply that anything should be hardcoded, I’m sorry if I’m unclear or imprecise. At a high level, I think it would be useful if this crate made the XDG standard directories available on macOS so developers had the option of using them in applications that are not exclusively macOS native applications.

For me, the divergence in paths only matters for the same user. The issue is that I may have both a Linux laptop and a Mac laptop, but on one my config files go to ~/.config/PROGRAM by default while on the other they go to ~/Library/Application Support/PROGRAM. This doesn’t happen on programs I listed above, such as fish and hub, but it does happen for programs using this crate.

As far as $XDG_CONFIG_HOME, I would expect the behavior to be the same on all platforms surfacing the XDG defaults, so I’d expect $XDG_CONFIG_HOME to work the same on Mac as on Linux when talking about surfacing directory paths using this crate. If someone is sharing their fish.config between multiple computers and multiple OSes, I don’t think it’s unreasonable that they’d set this config variable on Mac to use custom directories if they were also doing so on Linux. I think it would be more surprising if a program used the XDG defaults on Mac but didn’t check any env variables, so I’d hope that it would be used regardless of platform.

[tesuji]

I marked the two comments above as resolved since they didn't discuss this issue.

Thanks for opening this issue. I've away from my computer for a while. I'm sorry for the late response here.

About exposing the XDG conventions on macOS, I think the same could apply to Windows. I don't personally against exposing XDG conventions, but there are some API design issues should be resolved.

I have some questions to ask:

• Do you have any proposed APIs to resolve this issue?
• Which crates (dirs-sys, dirs, directories) will change?

[xmo-odoo]

Is this really a concern? If you want to follow xdg on systems which don't care for it, why not just use the xdg crate? It does xdg everywhere and specifically and wilfully only that.

[OJFord]

It's often a concern for users - search 'mac xdg' in some projects' issue trackers and you'll find plenty of spirited debate.

Mostly I've found that even when the maintainers really want ~/Library/Application\ Support/appname/Resources/MacOS or whatever it is, they're willing to settle for respecting $XDG_CONFIG_HOME if explicitly set (i.e. just not the $HOME/.config fallback).

I don't think there's a good reason to object to that, the best is perhaps complexity, not wanting to take on the additional code path; but the way to solve that is to push it up into projects like dirs_next and not worry about it.

[poliorcetics]

I'm a user of several projects using directories-next. Several respect the XDG_CONFIG_HOME env var if set and fall back to the default macOS dir if not. That's something I like and would like to be available by default in directories-next (or lower in the chain) so that I can have all my config file in $XDG_CONFIG_HOME and easily add them to a git for my dotfiles and such.

[utkarshgupta137]

I've had a brief conversation with @Dylan-DPC who is one of the owners & I think he is on board with addressing this issue. I hope we can move past the why & move forward with discussing the how of this problem. If you're not convinced that it is an issue, please read this once: https://github.com/dirs-dev/directories-rs/issues/47#issuecomment-1464129466. If you use macOS & don't agree with my reasoning (or the plethora of other issues linked in my comment), please feel free to share them.

On to the solutions: I have created #64, #65, #66 to bring these forks in sync with the original crates. Once they're merged, we can have a release so that the remaining users of the original crates can migrate to these forks, without any breakage concerns. After that, we need a way for exposing the XDG convention paths on Mac & Windows as well. I think any changes to the existing API (in terms of function signatures) are a non-starter since these crates already have millions of downloads & have 1000s of reverse dependencies. Similarly, changing the existing returned paths is also a bad idea.

I think that leaves us with 2 options:

• We add new APIs to expose XDG conventions on all platforms (we can have native & xdg modules):
  • Crates that already use some solution for following XDG conventions or new crates can just use the functions inside the xdg module & forget about it.
  • Having 2 sets of functions will make it easier for devs to implement a fallback logic in their crates if they want to avoid a breaking change or offer more flexibility.
  • Devs could also call functions from one module or the other based on some environment variable that they define.
  • Some devs may choose to just ask their users to manually migrate their confs.
  • This will probably require more input for existing crates in either adding a fallback logic or getting their users to migrate.
  • This would be very simple to implement, we just create a new module that uses platform-specific functions for home dirs & uses XDG-specific functions for everything else.
• We modify the existing functions to return locations based on the user's environment (& maybe some params by devs):
  • This would be a very simple change from the dev's point of view - they just have to update the package & add a note in their READMEs.
  • OTOH, we have to be careful about what environment variables we rely on for deciding the convention: should we only rely on XDG_CONFIG_HOME, even for cache/data/state/runtime? Or use all environment variables (IMO it would make the tools too brittle)? Would it be a breaking change for the users who've already defined the environment variables? Should we rely on some other environment variable (like RUST_USE_XDG) so that users have to explicitly opt in?
  • Although it might not sound like a good idea to depend on environment variables, it is not without precedent. ripgrep - which might be the most well-known Rust tool doesn't follow any conventions & asks users to set RIPGREP_CONFIG_PATH. Similarly, rustup, cargo, awscli, npm - all allow the user to change the default path using environment variables.
  • This would be more complex from an implementation point of view but still doable.

I think it would be great if we could offer both solutions to the developers & have 3 modules: native (current impl, will also be re-exported at the crate level for API compatibility), xdg, & hybrid. Since the hybrid solution requires too many decisions on our part or requires adding a bunch of config options, I think it would be better to start with just adding an xdg module & waiting to see if there is a desire to have a hybrid solution.

What are your thoughts @tesuji @Dylan-DPC?

[deanishe]

I think that leaves us with 2 options

What about keeping the API the same and using a build tag/target to enforce XDG conventions? (IDK Rust, so forgive me if that's a stupid suggestion.)

As I see it, on macOS it's basically a problem of UNIX programs not following UNIX conventions. 99% of the time, the library can do the right thing by knowing whether it's being used in a desktop application or a command-line one.

[utkarshgupta137]

I think that leaves us with 2 options

What about keeping the API the same and using a build tag/target to enforce XDG conventions? (IDK Rust, so forgive me if that's a stupid suggestion.)

As I see it, on macOS it's basically a problem of UNIX programs not following UNIX conventions. 99% of the time, the library can do the right thing by knowing whether it's being used in a desktop application or a command-line one.

That could work. I just think some crates might want to have access to both the XDG directories & the native directories (for fallback or migration purposes), so I didn't suggest that.

I've found the etcetera crate, which is doing most of what I suggested. Since the devs of this crate have not replied, I suggest people use it instead. I'll be contributing it for the foreseeable future.