Quod Libet doesn't match FreeDesktop directories specs

[GoogleCodeExporter]

I've tried Quod Libet 2.0 on Ubuntu 9.04 alpha 4
and it seems that Quod Libet places its configuration files in
/home/.quodlibet directory which doesn't match FreeDesktop directories specs :

The default for $XDG_CONFIG_HOME is $HOME/.config, the default for
$XDG_DATA_HOME is $HOME/.local/share. So all applications should look for
those environment variables and use those default values if the variables
are not set.

This is quite important since it's not possible to easily backup config
& data files if these files are not stored at the right place

See http://www.freedesktop.org/wiki/Specifications/basedir-spec
See also
http://ploum.frimouvy.org/?184-cleaning-user-preferences-keeping-user-data
(main post and comment#8)

Original issue reported on code.google.com by thibaut....@gmail.com on 24 Feb 2009 at 11:52

[GoogleCodeExporter]

Hello,

First I would like to add support for this request.

Implementing the Freedesktop specs for this means that two problems must be 
solved:
1. Discerning what is "user-specific data files" and "user-specific 
configuration".
2. Implementing the standard while remaining backwards compatible (i.e.: what 
happens
if a user upgrades from 2.0 to say 2.1, where 2.1 follows the Freedesktop XDG 
Base
Directory Specification).

FIRST ISSUE: WHAT GOES WHERE?

Quod Libet would need to put "user-specific data files" in 
$XDG_DATA_HOME/quodlibet/
or, if $XDG_DATA_HOME is not defined, in $HOME/.local/share/quodlibet/.
It would need to put "user-specific configuration" in 
$XDG_CONFIG_HOME/quodlibet/ or,
if $XDG_CONFIG_HOME is not defined, in $HOME/.config/quodlibet/.

But what is "user-specific data files" and what is "user-specific 
configuration"?
Configuration is anything that is defined by the user but for which there is a
predefined sensible default in the application. If it is lost, the user might to
configure a few things to get their previous setup back, but nothing of value 
will be
lost.
Data is anything that is produced by the activity of the user. For example: song
ratings, number of plays (if Quod Libet tracks those), playlists.

SECOND ISSUE: BACKWARD COMPATIBILITY

My take on this is that most applications that want to switch (from storing 
files in
$HOME/.appname/ to storing them in
$XDG_CONFIG_HOME/.appname/+$XDG_DATA_HOME/.appname) should do it like this: when
looking for data, it should first check if there is a $HOME/.appname/ directory 
(and
relevant files maybe). If it's there, it could either copy it to the new 
location, or
continue using it. If it's not there, it would then use the 
Freedesktop-compliant
locations.

It basically means that users upgrading won't lose configuration or data, and 
for new
users Quod Libet will follow the XDG Base Directory Specification.

(I have no opinion on whether the application, if the $HOME/.appname/ directory
exists, should keep using it, or copy its content to new locations and use 
those, or
move its content to new locations and use them. First solution seems simpler to
implement, the third one is maybe cleaner.)

Original comment by fverschelde on 7 Mar 2009 at 6:44

[GoogleCodeExporter]

Two more things:

1. There is a move towards supporting this Freedesktop specification. (Which is
logical if application developers agree that Freedesktop specs are "a good 
thing" and
strive to adopt them. I'm not sure if there's a very strong consensus towards
Freedesktop specs, though.) Right now most applications still use a 
$HOME/.appname
directory, or worse (VMWare uses a $HOME/vmware directory, annoying as hell). 
But
things are on the move. Applications that support this spec since at least
summer/fall 2008 include VLC, Totem, Sound Juicer, Transmission and others.

2. I played with the Quod Libet source (from a SVN checkout). From my 
experiments,
very basic support for the Freedesktop spec implies:
- removing the USERDIR constant defined in quodlibet/const.py;
- replacing it with two constants, for instance USERCONFDIR and USERDATADIR;
- replacing every occurrence of USERDIR in quodlibet source with either 
USERCONFDIR
and USERDATADIR.

I did just that (around 25 occurrences of USERDIR to replace), using those 
constants:

USERCONFDIR = os.path.join(HOME, ".config", "quodlibet")
USERDATADIR = os.path.join(HOME, ".local", "share", "quodlibet")

Which means I'm not checking whether there is a $HOME/.quodlibet directory 
already,
or doing fancier stuff (I'm not a developer, I'm just learning a bit of Python 
but
mostly for web development with Django). Well, I tried this and it worked fine.

With my limited knowledge of both Python and Quod Libet, I should be able to 
submit a
patch for implementing the first solution I suggested (checking if 
$HOME/.quodlibet
exists and use it if it does, and use the standard directories otherwise). I 
could
also ask for some information from people who understand the Freedesktop spec 
better
than I do, so that I may know where different data should go.

Tell me what you think. :)

Original comment by fverschelde on 7 Mar 2009 at 9:24

[GoogleCodeExporter]

Imho, before any coding someone should decide which files
go in which directory.

GNOME wiki quote:
"if you remove the .cache folder, the user will not notice it except maybe
for performance (might be huge performance). If you remove the .config folder,
the user will see all the preferences reset to the default but without any
data loss. In fact, it would not be a problem if the user didn't customize
his desktop."

The simplest, less error prone migration would be: only use the new schema
if there is no %HOME/.quodlibet or if there is a $XDG_CONFIG_HOME/.quodlibet
directory.

Here is a list of all used files in QL:
01) All plugin files in XXX/plugins/(playorder|songsmenu|editing|events)
02) Dump/MiniDump files
03) accels file
04) queue file
05) songinfo file
06) control file
07) config file
08) current file
09) songs file
10) logs folder (*.log files in XXX/logs/)
11) list folder:
    tagpatterns / tagpatters.saved
    renamepatterns / renamepatterns.saved
    queries /queries.saved
12) stations file
13) playlists file
14) browsers folder (containes py/pyc files)
15) feeds file
16) devices file
17) cache file

The directories could be different if he environmen variable is set...
I guess this is right:

DATADIR = os.path.join(os.getenv("XDG_DATA_HOME", HOME, ".local", "share"), 
'quolibet')
CONFIDIR = os.path.join(os.getenv("XDG_CONFIG_HOME", HOME, ".config")), 
'quolibet')
CACHEDIR = os.path.join(os.getenv("XDG_CACHE_HOME", HOME, ".cache")), 
'quolibet')

Original comment by reiter.christoph@gmail.com on 9 Mar 2009 at 10:11

[GoogleCodeExporter]

Found some more.. here is a first attempt.
Feedback please.

01) All plugin files in XXX/plugins/(playorder|songsmenu|editing|events) -> data
02) Dump/MiniDump files -> data
03) accels file -> config
04) queue file -> data
05) songinfo file -> data
06) control file -> data
07) config file -> config
08) current file -> data
09) songs file -> data
10) logs folder (*.log files in XXX/logs/) -> data/logs
11) list folder:
    tagpatterns / tagpatters.saved
    renamepatterns / renamepatterns.saved
    queries /queries.saved -> config/list
12) stations file -> data
13) playlists file -> data
14) browsers folder (containes py/pyc files) -> data/browsers
15) feeds file -> data
16) devices file -> data
17) cache file -> cache
18) cover.png -> data (notify plugin)
19) album_pattern -> config (album browser)
20) scrobbler_cache -> cache (qlscrobbler plugin)

Original comment by reiter.christoph@gmail.com on 9 Mar 2009 at 2:55

[GoogleCodeExporter]

Christoph, thanks for that work. I had missed the “cache” part.

Ploum, cited in Thibault's opening message, has a follow-up post which presents 
the
distinction between user data, user configuration and cache in plain English:
http://ploum.frimouvy.org/?207-modify-your-application-to-use-xdg-folders

Here is some feedback for your list of Quod Libet data. I must say i'm not a
developer per se, and don't know a lot about Quod Libet, though. ;)

Note 1: when my choice differs from yours, i indicate yours in parenthesis.
Note 2: sometimes i'm not sure what a file is about, so i only indicate your 
choice.
I'm not on a Linux computer right now so i can't check those files.
Note 3: i don't write subdirectories in my feedback. Of course, once a category 
(user
data/user config/cache) is chosen, the files can go in subdirectories like 
"list" or
whatever's needed.

01) All plugin files in XXX/plugins/(playorder|songsmenu|editing|events) -> 
CONFIG?
(data)
02) Dump/MiniDump files -> not sure what it is (data)
03) accels file -> not sure what it is (config)
04) queue file -> CONFIG (data), whereas default setting is "empty queue"
05) songinfo file -> not sure what it is (data)
06) control file -> not sure what it is (data)
07) config file -> CONFIG, obviously
08) current file -> CACHE (data)
09) songs file -> DATA, could have been CACHE if it did not contain metadata 
that
cannot be retrieved by refreshing the library (like song ratings, number of 
plays, etc.)
10) logs folder (*.log files in XXX/logs/) -> CACHE? (data/logs) since it's not 
user
data per se
11) list folder:
    tagpatterns / tagpatters.saved -> CONFIG
    renamepatterns / renamepatterns.saved -> CONFIG
    queries /queries.saved -> CONFIG
12) stations file -> DATA, actually could be config as well but if you only 
want to
backup your user data you'll probably want to backup this
13) playlists file -> DATA, actually could be config as well but if you only 
want to
backup your user data you'll probably want to backup this
14) browsers folder (containes py/pyc files) -> not sure (data/browsers)
15) feeds file -> not sure (data)
16) devices file -> CONFIG? (data)
17) cache file -> CACHE
18) cover.png (notify plugin) -> CACHE (data), like current file that's 
volatile data
19) album_pattern (album browser) -> CONFIG
20) scrobbler_cache (qlscrobbler plugin) -> CACHE

I'll have to revise/complete this.

ON A SIDE NOTE: i guess a quick 'n dirty solution would be to move 
HOME/.quodlibet to
HOME/.config/quodlibet, and not to sort between user data, user preference and 
cache.
I can see this could be acceptable (though non conforming) for the following 
reasons:

- Quod Libet doesn't generate lots of user data. The only data that's clearly 
uer
data is playlists, and part of the songs database IF the user stores 
information in
the songs database that can't be retrieved from the files themselves. The main 
user
data quodlibet uses and even writes (through Ex Falso) is music files, and 
those are
managed by the user in a non-hidden directory or other places.
- Quod Libet and its plugins use little cache (not sure how the cache file 
itself is,
but i think it's quite small), so going to the extent of using a specific cache
folder might be overkill.

Original comment by fverschelde on 5 Apr 2009 at 4:54

[GoogleCodeExporter]

Since QL won't be switching names, this change should not be rushed; I'd prefer 
to
wait on this one until after the rather large list of changes in 2.1 have been 
tested
and released.

Original comment by steven.strobe.cc@gmail.com on 18 Jun 2009 at 8:15

• Added labels: Milestone-longterm

[GoogleCodeExporter]

Long term sounds good if that means work on this issue is scheduled and it gets 
done
eventually. :)

Original comment by fverschelde on 18 Jun 2009 at 11:41

[GoogleCodeExporter]

I'd really like to see this change. this issue exist for almost 2 years but not 
implement...

I don't think the config files are that important that application can not 
switch the restore place.

This can be done in a major version change. just a warning window would be good 
enough. if there is any problem, user can easily solve them by google it. 

Acturally i think change like this should implement as soon as possible, cuz 
this application is more and more widely used.

And apps like AWN, cairo-dock, rhythmbox didn't wait that long.

Original comment by jason5...@gmail.com on 23 Nov 2010 at 1:44

[GoogleCodeExporter]

Patch: Simply make ~/.quodlibet to $XDG_CONFIG_HOME. use glib way to do this 
job will be much easier.

Original comment by jason5...@gmail.com on 23 Nov 2010 at 2:15

Attachments:

• xdg.patch

[GoogleCodeExporter]

That patch makes it impossible for me to have ~/.quodlibet, which is what I 
want.

Original comment by joe.wreschnig@gmail.com on 23 Nov 2010 at 5:02

[GoogleCodeExporter]

Is there a big different to access ~/.quodlibet than ~/.config/quodlibet? if 
you want you can make soft link.

Do you like to see a huge number of hiden files when enable show hiden files in 
filemanager or use tab to complete command? It's in your $HOME dir so you have 
to see them a lot no matter you want to or not.

Original comment by jason5...@gmail.com on 23 Nov 2010 at 5:25

[GoogleCodeExporter]

I don't see the value in having half my files in ~/.config and half my files in 
~/. I don't find value in change for the sake of change. I don't find any 
arguments for ~/.config persuasive. (Is the only argument "We can crappy up 
this folder instead of that folder?")

I've been using Unix for 18 years now and I've never been bothered by having rc 
files in my home directory, at least not in any way that moving them all to 
another directory would help, and certainly not in a sense that moving some 
random subset to another directory would help.

Original comment by joe.wreschnig@gmail.com on 23 Nov 2010 at 7:35

[GoogleCodeExporter]

One of the advantage of move is to allow users to easily backup their system 
choosing to restore only their data or their config or both easily.
It can't be done easily at the moment.
It will be very easy to create backup tools that will provide that kind of 
choice then.
Actually, it is a move to allow users to better handle its files to my mind.
it is definitely worth it.

Original comment by thibaut....@gmail.com on 23 Nov 2010 at 10:19

[GoogleCodeExporter]

example: you can easily clean up cache files by delete ~/.cache dir, if every 
apps match XDG standards, or it would be really hard for a user to do taht.

besides, there is no advantage to put all files into $HOME, either. so why not 
just match XDG standards?

@thibaut.bethune, actually it woundn't be done easily at any moments, and if 
more and more user using this, it would be harder and harder to handle all the 
situation.

Original comment by jason5...@gmail.com on 24 Nov 2010 at 4:56

[GoogleCodeExporter]

Why is there this arbitrary distinction between "config" and "data"? When am I 
going to be in a situation where I don't want to back up both? It seems like a 
very un-Unix split also - what's my ~/.emacs? Surely it's a configuration file, 
but it contains more code than many programs!

Why is there no way to configure all XDG-aware apps to behave as applications 
have for decades? (There's trivial ways to do this, like using XDG_CONFIG_ROOT 
as a string prefix rather than a parent directory, but the standard forbids 
them.)

Why did XDG not standardize on existing practice, as was the common way for 
e.g. the LSB and other open and closed standards bodies to work?

Changes like this contribute to nothing but bitrot of otherwise stable software 
and unnecessary code churn for active ones. This whole thing reminds me of how 
GConf was going to be the be-all-end-all solution for configuration storage 
some years ago, and it was vitally important for application authors to support 
that.

Original comment by joe.wreschnig@gmail.com on 24 Nov 2010 at 12:43

[GoogleCodeExporter]

Traditionally programs don't make a clear distinction between config and data 
(and cache) files therefore the move can't be automated.
It would have been possible if there still were /.config and /.data subfolders 
in any ~/.program folder but it is not the case.
The spec asks developpers to make that distinction obvious.
That lloks like a progress to me since a better classification will allow new 
developpements that weren't possible before.

Original comment by thibaut....@gmail.com on 24 Nov 2010 at 1:33

[GoogleCodeExporter]

No, traditionally programs don't make a clear distinction between config and 
data because they're not different, and to pretend they are different is to 
pretend programmable, extensible, really configurable applications don't exist.

Traditionally, programs don't write cache files, because cache files only 
matter for the small set of programs that download large resources over slow 
links. (I think the only thing QL caches are thumbnails - which there's a 
separate fd.o spec for, and it doesn't use ~/.cache either. Great thing about 
standards, etc.) (The scrobbler cache file is identified as a cache above, but 
it's not, it's data.)

And there's no place in the XDG spec for QL's control, songinfo, cover.png, and 
other interface files. Their lifetime would be appropriate for a cache folder, 
but they're not cache files.

This is change, not progress.

Original comment by joe.wreschnig@gmail.com on 24 Nov 2010 at 4:28

[GoogleCodeExporter]

If my memory is correct, Rhythmbox treats covers as cache since the application 
automatically downloads them from the Internet.
Therefore if you delete them, they won't miss you since the application will 
bring them back. It will only take some more time : like caching. No data will 
be lost from that point of view.

Original comment by thibaut....@gmail.com on 24 Nov 2010 at 10:12

[GoogleCodeExporter]

Quod Libet writes cover files alongside the audio files, where they should be. 
(Or in metadata tags.) The cover cache is to avoid loading sometimes extremely 
large files off slow media and rescaling. That's identical in purpose to the 
thumbnail cache, a standard for which already exists, and is not in ~/.cache. I 
don't understand why you brought that up though.

Original comment by joe.wreschnig@gmail.com on 24 Nov 2010 at 10:58

[GoogleCodeExporter]

There is indeed a conflit between the 2 specs : Thumbnail Managing Standard and 
FreeDesktop directories.

I only wanted to help by answering "there's no place in the XDG spec for QL's 
control, songinfo, cover.png, and other interface files. Their lifetime would 
be appropriate for a cache folder, but they're not cache files."

If you don't think that this spec can give the user the ability to control its 
configuration then i guess you should mark this bug as wontfix.
Since you are the developer, you decide : no offense :-)

Original comment by thibaut....@gmail.com on 24 Nov 2010 at 11:11

[GoogleCodeExporter]

Because Google Code is going away [0] we've moved this bug tracker to GitHub.

This issue is now available at

https://github.com/quod-libet/quodlibet/issues/138

In case you are still interested in this issue and have a GitHub account, you
can subscribe to the new issue using the "Subscribe" button on the right hand
side of the page.

[0] http://google-opensource.blogspot.co.at/2015/03/farewell-to-google-code.html

Original comment by reiter.christoph@gmail.com on 16 Mar 2015 at 11:19

• Changed state: Moved