Closed bos closed 1 year ago
(Imported comment by @dcoutts on 2010-05-12)
I'm not quite sure why you say it is non-standard. It's the standard unix place to put per-user application files. Where would you consider to be a standard location?
You can use:
cabal --config-file=... [command] [flags]So it should work to use a shell alias, something like
alias cabal="cabal --config-file=..."So really the bug here is not that it's hard coded (since it's not) but that these global flags are not easily discoverable. We don't list them all in the global --help so as not to clutter things, but then we need a way to let people know that there are extra global options and provide a way to get the help for them via some extended --help thing. So let us know about:
(Imported comment by guest on 2010-05-13)
Sorry for neglecting to answer right away, I forgot about this bug for a few days because I didn't bother to jump through the hoops to be able to receive change notification.
If by standard you mean de facto just everyone's been doing it like that, then I agree. And since indeed everyone's been doing that, I end up with hundreds of dotdirs in my home directory. This sucks because their content is essentially unstructured. The problem is that I want to treat files differently according to their purpose; and there are not just application files. There are also configuration files, they are put under version control. There are cache files, they are excluded from backup and indexing.
You know that a software package in Linux is not just dumped into a single directory somewhere, but the files are sorted into directories which each have a certain meaning. FHS governs the semantics for the system layout, FDO's XDG base directory specification governs this for a user's directory. These are real standards, and they enable all sorts of interoperability that would not be possible with unstructured directory contents.
Whether you want to adopt the spec in order to sort files into their appropriate XDG location by default is up to you. Thanks to your hint about --config-file I was able to move away the last file and delete .cabal for good, so certainly a compliant layout is already possible (through careful configuration, as I said before) if a user wants it.
(Imported comment by @dcoutts on 2010-05-17)
I thought you might say the XDG spec. Yes I realise it's a "real" standard as opposed to a de facto standard. I must admit I'm slightly sceptical about the attempt to redefine where apps keep their per-user info. So far, very few apps seem to follow this new spec (mainly a few GNOME desktop bits). I will be surprised if they ever manage to get more than the two main desktops to move their dot files.
That said, if the world is going to move that direction then we don't want to get in the way. We should make it easier for users or distros to change where the per-user files go by default.
Thanks for confirming it is at least possible (if not convenient).
(Imported comment by guest on 2010-05-17)
we should open another ticket on providing extended help for the global flags
This is now tracked in #690.
(Imported comment by @dcoutts on 2010-05-17)
Hmm, I notice the XDG base directory specification does not say anything about program executable files or library files, though the default naming scheme sort-of suggests ~/.local/bin and ~/.local/lib.
(Imported comment by Syzygies on 2010-05-17)
I found this ticket while researching precedents for a related question.
For various reasons I like to build multiple versions of GHC side by side: Migration between major releases, testing candidate releases. For torture-testing new hardware, building many copies of GHC at once is better than e.g. the usual prime testing. I once literally caused a server power supply to smoke this way; my friend's hardware vendor was in over their head. They took my GHC test in-house to minimize delivery cycles.
A related piece of de facto convention comes up here: It is crazy to sprinkle pieces of a system like GHC into separate global locations like /usr/local/bin, /usr/local/share, unless one will only ever use one version, with no desire to look under the hood or move the installation. In other words, I have no issue with Linux or OS X shipping this way, but major additions should be localized by default. Some are, e.g. TeX. GHC should.
Fortunately, to avoid Siamese twin installations with accidentally shared parts, one can use the ./configure --prefix option while building GHC. Then, it becomes apparent that "cabal install" is the remaining holdout to ideal version localization. There is a perfect storm of tiny factors keeping Cabal from gracefully localizing. I wanted to elicit advice before contributing a fix; this ticket seems to have touched on a key issue.
I did not know about the "--config-file=..." command-line option; I will start using it in scripts. However, we need a better default for global installs.
In /usr/local I now use a symbolic link from ghc to one of the directories ghc-6.10.4, ghc-6.12.1, ghc-6.12.2, ghc-6.12.3-rc1. It then appears that I built GHC using "--prefix=/usr/local/ghc". My ~/.cabal/config file includes lines like "remote-repo-cache: /usr/local/ghc/packages". This works, but I've had amusing debates with unix experts on why my symbolic link is a terrible solution. Basically, when I change this link I pull the rug out from under any ongoing process that wants to see a consistent environment.
Environment variables are one "right" way to handle this. In short, a symbolic link is like changing state, while environment variables are like functional programming.
However, not only does .cabal/config fail to allow external environment variables, it doesn't allow forms like one sees in its own comments. (Who reads documentation? We all read sample code.) For example, even though I see the comment "-- bindir: $prefix/bin" I cannot write "remote-repo-cache: $prefix/packages". I would prefer to be able to write anything that would evaluate correctly in a shell to an absolute path, e.g. which ghc or perl -pe ... and so forth.
Ideally, a global install would by default use a global location for the Cabal config file, but we slam back into the original problem: By default the global install of GHC did not use the --prefix option, so there is no well-defined location parametrized by the version of GHC. On the other hand, this isn't so bad, e.g. put everything in ../share/hackage.haskell.org relative to the currently visible ghc. Saves bandwidth caching packages; if someone hasn't bothered to build ghc using --prefix, they won't mind the Siamese twin effects with Cabal either.
The simplest change that would help my issue would be to allow both absolute and relative paths in .cabal/install, with a relative path relative to the location of which ghc. This would "just work" in simple cases, and allow competing processes to coexist, that chose different versions of ghc via a custom $PATH alone, with no other flags needed. If this change was accepted, I wouldn't mind leaving my config file in ~/.cabal. That location is only mildly annoying; as has been noted, we've already lost the war on keeping that area uncluttered.
However, it just shouldn't matter which authorized user works on a global install. Relying on ~/.cabal/config for a global install should be viewed as a bug, as it prevents several admins from cooperating in consistently maintaining GHC on a shared system.
(Imported comment by @dcoutts on 2010-05-28)
Replying to Syzygies:
A related piece of de facto convention comes up here: It is crazy to sprinkle pieces of a system like GHC into separate global locations like /usr/local/bin, /usr/local/share, unless one will only ever use one version, with no desire to look under the hood or move the installation. In other words, I have no issue with Linux or OS X shipping this way, but major additions should be localized by default. Some are, e.g. TeX. GHC should.
You can already do this if you want to. You just set the --prefix to be what you want, e.g. instead of using /usr/local as the prefix, use /opt/ghc/6.12.1 or whatever, then you'll get paths like /opt/ghc/6.12.1/bin, /opt/ghc/6.12.1/lib, /opt/ghc/6.12.1/share etc etc.
Fortunately, to avoid Siamese twin installations with accidentally shared parts, one can use the ./configure --prefix option while building GHC. Then, it becomes apparent that "cabal install" is the remaining holdout to ideal version localization. There is a perfect storm of tiny factors keeping Cabal from gracefully localizing. I wanted to elicit advice before contributing a fix; this ticket seems to have touched on a key issue.
I did not know about the "--config-file=..." command-line option; I will start using it in scripts. However, we need a better default for global installs.
The "default" default, should follow global conventions, meaning using /usr/local. Again you can override the prefix and all other install directories.
It's true that there is not currently a sensible way to have a global cabal config file, in the style of /etc/cabal or something like that. Following the XDG spec in that regard might make some sense.
Environment variables are one "right" way to handle this. In short, a symbolic link is like changing state, while environment variables are like functional programming.
You mean you would like an environment variable to set the location of the Cabal config file? You can set the $CABAL_CONFIG, it acts like setting the "--config-file=..." option.
However, not only does .cabal/config fail to allow external environment variables, it doesn't allow forms like one sees in its own comments. (Who reads documentation? We all read sample code.) For example, even though I see the comment "-- bindir: $prefix/bin" I cannot write "remote-repo-cache: $prefix/packages". I would prefer to be able to write anything that would evaluate correctly in a shell to an absolute path, e.g. which ghc or perl -pe ... and so forth.
The prefix is something that makes sense in the context of an installation. It is not clear that basing the cache location on that makes sense. It means for example that it will not find the default cache if you override the --prefix on the command line. Perhaps what you really want to do is to specify the default prefix and cache dirs in the config file based on some other variable, rather than having the cache dir follow the prefix.
Ideally, a global install would by default use a global location for the Cabal config file,
Do you mean an install run as root, or do you mean an install run as a user? Or do you mean that you'd like to be able to have a shared/global set of connfig defaults that can be overridden in per-user config files?
As a user when you run cabal install --global --su-cmd=sudo you're using your own config, but installing to a global location (the exact prefix etc is set in the config file). It does the build as user and installs as root.
but we slam back into the original problem: By default the global install of GHC did not use the --prefix option, so there is no well-defined location parametrized by the version of GHC.
You can either use versioned ghc binaries, (called ghc-6.10.4, ghc-6.12.1 etc) in a shared prefix like /usr/local or as I described above you can use separate prefixes for different ghc installations. Note that by default, cabal installs libs in dirs including the ghc version, so it all works. As mentioned before you have complete control over the prefix (and other detailed layout of bindir etc) so you can put files where you like, according to some ghc version scheme. You can use the compiler id in the cabal prefix for example.
The simplest change that would help my issue would be to allow both absolute and relative paths in .cabal/install, with a relative path relative to the location of which ghc.
Honestly I don't think that is sensible. Allowing people to use more variables so that you can specify a prefix that happens to correspond to your ghc installation is one thing, making that the meaning for relative paths is rather another. Usually you do not want to install things into the ghc tree, so using relative paths would encourage that dangerous behaviour.
However, it just shouldn't matter which authorized user works on a global install. Relying on ~/.cabal/config for a global install should be viewed as a bug, as it prevents several admins from cooperating in consistently maintaining GHC on a shared system.
I don't understand this bit. Perhaps you can explain what config you would like to use for global installs. Where would it live? Do you assume global installs are only performed by root?
I think it would help if you could explain what you're actually trying to achieve. The mailing list might be the better place for that discussion.
(Imported comment by @kosmikus on 2010-05-28)
Moving to 0.16 milestone so that we look at it again sometime soon, but I think this is a candidate for | (meaning that without external concrete patches or suggestions, we probably won't do it).
What is planned now about FreeDesktop XDG basedir specification for cabal?
http://ploum.net/post/207-modify-your-application-to-use-xdg-folders https://live.gnome.org/GnomeGoals/XDGConfigFolders http://standards.freedesktop.org/basedir-spec/latest/
According to XDG Base directory specification, cabal should not have its own folder (.cabal) anymore User data should go into $XDG_DATA_HOME (which default to ~/.local/share), user preferences should go into $XDG_CONFIG_HOME (which default to ~/.config) and cached data should go to $XDG_CACHE_HOME (which default to ~/.cache). More details at : http://ploum.net/post/207-modify-your-application-to-use-xdg-folders https://live.gnome.org/GnomeGoals/XDGConfigFolders
Full specification can be found at: http://standards.freedesktop.org/basedir-spec/latest/
The Freedesktop.org XDG base directory specification have good de facto adoption. It has been adopted by:
I think that cabal should use same locations than the vast majority of Desktop environment and applications.
There are real advantages of following this specification :
In general, I tend to agree with these advantages.
However, the XDG spec seems to be designed with typical desktop application in mind, not compilers or package managers. As @dcoutts said above, it doesn't specify where the binaries, libraries, documentation should go. Are there any examples of programming language implementations that use this scheme?
Also, should we try to follow the same spec on Windows as well? (I guess there are no technical reasons not to do so.)
Users binaries, documentation and libraries are users data and thus should go to $XDG_DATA_HOME (which default to ~/.local/share),
"Windows have the Local/Roaming split which could be roughly equated to putting everything in either $XDG_DATA_HOME or $XDG_CACHE_HOME like most Humble Bundle Linux game ports do."
Users binaries, documentation and libraries are users data and thus should go to $XDG_DATA_HOME (which default to ~/.local/share),
Yes, that's exactly what I meant by the desktop application oriented mindset.
Putting binaries, libraries and docs in one directory is a downgrade from the scheme that we have now.
Or would it be $XDG_DATA_HOME/bin
etc.? Then the full path is ~/.local/share/bin
, which looks weird and complicated.
I think it is better to avoid users specific binaries, libraries and doc and share these files between all users.
Whether it is better or not is out of the scope of this issue, I believe.
There is no $XDG_DATA_HOME/bin etc but you are free to use $XDG_DATA_HOME/cabal/bin etc
So that would be ~/.local/share/cabal/bin
, which looks rather disappointing to anyone who ever cares about what these paths are.
I think at this point it would be more realistic for someone who cares to create a simple cabal-install wrapper that would conform to XDG. Then release it and see if it gains popularity. If it does, it would be a convincing argument to switch to that scheme by default.
Instead of conforming to XDG by default I'd be happy with cabal-install reading an environment variable telling cabal where its config is. Environment variables are much more persistent than some kind of flag+alias for that. Then you can configure all other directories through config anyway.
EDIT Ignore me, reading an issue I found $CABAL_CONFIG
. I'd be great if it was documented though.
EDIT2 It'd be great if cabal config evaluated external environment variables as well.
Personally I think cabal-install should follow XDG here which implies ${XDG_CONFIG_HOME:-$HOME/.config}/cabal
for {config,world}
and ${XDG_DATA_HOME:-$HOME/.local/share}/cabal
for {bin,lib,logs,share}
and ${XDG_CACHE_HOME:-$HOME/.cache}/cabal
for {packages,setup-exe-cache}
. It can then, separately, symlink binaries to a configurable location defaulting to $HOME/.local/bin
(no XDG), which isn't in any official standard I'm aware of but is used by Python (like PREFIX=$HOME/.local
) and is on the user $PATH
by default in at least Fedora.
I have no idea what those paths should map to on Windows systems but presumably that's a solved problem.
You realise that you guys can actually change the XDG_*
environment variables? XDG_DATA_HOME
doesn't have to be HOME/.local/share
. We'll get there eventually, compilers are one of the few remaining bastions supporting the UNIX v2 bug that is "dotfiles"
@Earnestly Exactly. That is the default but can be changed as you like. So, @nagisa, $CABAL_CONFIG
would be the exact same thing as the XDG variables.
Another problem with using a unique environment such as CABAL_CONFIG
is we
add to our already bloated environ
s. For reference here are a few:
export SLRNHOME=$XDG_CONFIG_HOME/slrn
export _RR_TRACE_DIR=$XDG_DATA_HOME/rr
export CARGO_HOME=$XDG_DATA_HOME/cargo
export GNUPGHOME=$XDG_CONFIG_HOME/gnupg
export INPUTRC=$XDG_CONFIG_HOME/inputrc
export RLWRAP_HOME=$XDG_DATA_HOME/rlwrap
export LYNX_CFG_PATH=$XDG_CONFIG_HOME/lynx
export DVDCSS_CACHE =$XDG_CACHE_HOME/dvdcss
export TIGRC_USER=$XDG_CONFIG_HOME/tig/tigrc
export WEECHAT_HOME=$XDG_CONFIG_HOME/weechat
export PERL_CPANM_HOME=$XDG_CACHE_HOME/cpanm
export GIMP2_DIRECTORY=$XDG_CONFIG_HOME/gimp
export ELINKS_CONFDIR=$XDG_CONFIG_HOME/elinks
export CABAL_CONFIG=$XDG_DATA_HOME/cabal/config
export HTTPIE_CONFIG_DIR=$XDG_CONFIG_HOME/httpie
export LESSHISTFILE=$XDG_CACHE_HOME/less/history
export XAUTHORITY=$XDG_RUNTIME_DIR/x11/xauthority
export XCOMPOSEFILE=$XDG_CONFIG_HOME/x11/xcompose
export GUILE_HISTORY=$XDG_CONFIG_HOME/guile/history
export VIMPERATOR_RUNTIME=$XDG_CONFIG_HOME/vimperator
export PENTADACTYL_RUNTIME=$XDG_CONFIG_HOME/pentadactyl
export NOTMUCH_CONFIG=$XDG_CONFIG_HOME/notmuch/notmuchrc
export GTK2_RC_FILES=$XDG_CONFIG_HOME/gtk-2.0/settings.ini
export TERMINFO_DIRS=$XDG_DATA_HOME/terminfo:/usr/share/terminfo
export UNCRUSTIFY_CONFIG=$XDG_CONFIG_HOME/uncrustify/uncrustify.cfg
These are just the ones I use, there are much more. Clearly this is not sustainable, each export adds to the total memory consumption of -every- child process on the machine.
Here are some more https://wiki.archlinux.org/index.php/XDG_Base_Directory_support#Partial
So no improvements here? Another reason for adopting the XDG scheme is that (some) backup tools know about these and already exclude e.g. .cache from backups. I'm now hunting around to see which of its 3GiB cabal actually needs :-(
@rkraneis I think this mostly needs somebody to take the lead: design a reasonable migration/fallback scheme, and implement it.
I was browsing this topic researching what the XDG file location concept says about binaries - not to much as it seems - and I found that @Earnestly wrote a few years back:
... We'll get there eventually, compilers are one of the few remaining bastions supporting the UNIX v2 bug that is "dotfiles"
however the original link was a Google+ one that has now been expunged from the Interweb thingummyjig - but I tracked down a wayback machine archive and fitted it into this post. HTH a tiny bit...
Until PRs are done, I updated the entry in the Arch wiki with easy environment variables ;) https://wiki.archlinux.org/index.php/XDG_Base_Directory#Partial
I'm idly poking at this now. Did someone ever draft a design or migration path? Let me suggest the following:
$XDG_CONFIG_HOME/cabal/config
.cabal install
binaries go in $HOME/.local/bin
(from the systemd file-hierarchy spec).$XDG_CACHE_HOME/cabal
).This might break people, of course. So if CABAL_DIR
exists, then the current behaviour will be maintained (everything in CABAL_DIR
). Further, as a (hopefully temporary) migration path, if $HOME/.cabal
exists, then that will interpreted as CABAL_DIR
being set to $HOME/.cabal
.
I have no idea what the situation is on Windows. Maybe current behaviour should just be preserved there.
Something not mentioned previously. Users are able to configure using CABAL_CONFIG
and having user specific paths in their cabal.config
. E.g. mine looks like:
repository hackage.haskell.org
url: http://hackage.haskell.org/
remote-build-reporting: anonymous
remote-repo-cache: /cabal/packages
write-ghc-environment-files: always
build-summary: /cabal/logs/build.log
extra-prog-path: /cabal/bin
installdir: /cabal/bin
logs-dir: /cabal/logs
store-dir: /cabal/store
symlink-bindir: /cabal/bin
world-file: /cabal/world
install-dirs user
prefix: /cabal
Changing defaults is more or less a communication issue (EDIT: i.e. not a technical problem)
I don't think the code for that will need to be modified. XDG migration is only about the defaults, anyway. From hacking a bit on the code, I think the largest unknown challenge is that there seems to be some parts of the code that assume a single "root" cabal
directory from which other paths can be computed. I'm not yet sure how deep that goes.
Shouldn't go deep, as all used paths should be configurable. Their defaults are computed using CABAL_DIR
.
Everything else goes in $XDG_CACHE_HOME/cabal
If the cache is cleaned, this can break projects and installed executables, so I'd suggest XDG_DATA_HOME instead
I don't have a strong opinion on that. I tried looking at whether XDG has firm guidance on what distinguishes "data" from "cache", but I couldn't quickly find any. My intuition was that "cache" is for things that can be recreated, while "data" is for things that might be unique.
@fgaz XDG_DATA_HOME
is wrong, as it could be shared between machines, thus XDG_STATE_HOME
.
First of all let's get more granular: packages
can for sure go in cache
, and maybe logs too, while store
is the problem.
on the other hand
I'd say that it's better to err on the safe side and put it into datastate, but I don't have a strong opinion either
This discussion is split now between this and #7386. Let's decide where to discuss.
oof I didn't see the comment on the pr yet, sorry
There's a lot of these XDG issues (e.g. also #1857). I'd suggest closing all but one (say, this one), and then keep the design discussion here (not in the PRs, which might be closed or replaced if a better one shows up).
store-dir: $home/foobar
- it just happens to mention XDG. (If implemented, it would make default config user-independent, which would be great - but the issues can be resolved independently).I see that XDG_STATE_DIR is only a proposal by debian for now: https://wiki.debian.org/XDGBaseDirectorySpecification
Do we want to use a specification that is not stable yet? I say yes, it solves a real need.
EDIT: nevermind, my search skills are weak. that page looks outdated and the spec now includes state: https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html
Is the conclusion that using XDG_STATE_DIR
is desirable? If so, directory
needs to be patched first.
I see it has already been reported: https://github.com/haskell/directory/issues/119
Yes, I think that's the conclusion. I also agree it's a good idea to discuss spec here and implementation in the PR. I will close any duplicate issues and link here from any related issues.
I submitted a patch to directory
for adding support for XDG_STATE_HOME
. We'll see how that goes. The draft cabal
design as I understand it is then:
$XDG_CONFIG_HOME/cabal/config
.cabal install
binaries go in $HOME/.local/bin
(from the systemd file-hierarchy spec).~/.cabal/packages
and ~/.cabal/logs
) go in XDG_CACHE_HOME/cabal
.~/.cabal/store
?) goes in $XDG_STATE_HOME/cabal
People with more Cabal experience can probably judge whether this is a sensible design. There's still two questions to answer:
1) What about Windows? The above will work on Windows because directory
supplies fascimiles of the XDG paths, but I'm not sure it's actually what users would want.
2) A migration path for people who have already set CABAL_DIR
or CABAL_CONFIG
. I think these variables should just take precedence over XDG, if set.
The CABAL_DIR
is being discussed at #7455 and I think we should stick to that or discuss there.
cabal install binaries go in $HOME/.local/bin (from the systemd file-hierarchy spec).
$HOME/.local/bin
is accepted in XDG spec: https://gitlab.freedesktop.org/xdg/xdg-specs/-/merge_requests/38
Note that stack also installs into this directory... I'm generally not a fan of this directory anymore. It has caused problems for ghcup as well to the point that I'd like to remove XDG support from it. As an example: if you'd enable XDG for both cabal and ghcup and then run cabal install --overwrite-policy=always --install-method=copy cabal-install
, then you'll cause confusion for ghcup by overwriting ~/.local/bin/cabal
.
But I'm sure many users already set this as their default installation directory, so yeah. But the (weak) guarantee that only a single tool installs into ~/.whatnot/bin
is a useful one.
A migration path for people who have already set CABAL_DIR or CABAL_CONFIG. I think these variables should just take precedence over XDG, if set.
Yes. ghcup
on windows sets CABAL_DIR
to a short path to avoid MAX_PATH issues. Following the XDG emulations will likely cause problems on windows.
@hasufell: thank you for the remarks. That's very valuable. Let's discuss how to avoid the problems.
There are at least two partially independent proposals here. One is XDG, the other is using the config file instead of environment variables (except CABAL_CONFIG
).
I think using $HOME/.local/bin
is preferable, even though it runs the risk of conflicts. Conflicts will always happen when you have a shared resource, but the Unix/Linux world is for better or for worse standardising on this directory. I don't think it's to the advantage of cabal to be different.
(I would also suggest cabal install
adopt common conventions from other tools, namely copying and overwriting by default, but that's unrelated to XDG).
I think using
$HOME/.local/bin
is preferable, even though it runs the risk of conflicts. Conflicts will always happen when you have a shared resource, but the Unix/Linux world is for better or for worse standardising on this directory. I don't think it's to the advantage of cabal to be different.
I'm not sure. All of the other XDG directories are usually used in a way where programs create a program specific subdirectory, exactly to avoid conflicts.
~/.local/bin
is the only one that breaks this.
Imagine all apps installing their data files into the same dir without subdirectories. Not a great idea.
The only reason this works for /usr/bin
is because there's only one program installing into it: your package manager.
My argument is not that $HOME/.local/bin
is a good design. My argument is that it's the de facto standard. It is used by build tools that are held in high regard by their respective user communities (e.g. cargo
, stack
), and complaints are few. Is this a fight that cabal
cares about?
(Imported from Trac #688, reported by guest on 2010-05-12)
I want to be able to tell cabal to read the configuration from a different place than $HOME/.cabal/config.
The ultimate goal is to get rid of the non-standard .cabal directory altogether, so far careful --package-db/--prefix etc. juggling served me well; config is the last remaining file there.