search

itchio / itch

🎮 The best way to play your itch.io games
https://itch.io/app
MIT License
2.36k stars 209 forks source link

Do not save apps/cache/downloads etc in ~/.config #1184

Open shunsei opened 7 years ago

shunsei commented 7 years ago

On linux ~/.config/itch/ contains all kinds of stuff that is not configuration files.

Please follow proper conventions and save cache files in ~/.cache/ and data files in ~/.local/share.

fasterthanlime commented 7 years ago

Electron apps treat ~/.config/[appname] as if they were C:/Users/[user]/AppData/Roaming/[appname] or ~/Library/Application Support/[appname], making Linux a special case would add a lot of complexity (and probably bugs), I'm not entirely convinced it's worth it.

Is the problem for you that you're backing up ~/.config, @shunsei ?

shunsei commented 7 years ago

Yes, there are multiple use cases this is interfering with.

  1. It should be possible to keep ~/.config/ under version control or back it up without backing up data files.
  2. It should be possible to free up disk space by cleaning up ~/.cache.
  3. It should be possible to use another partition for data files (eg. ~/.local/share).

And so on.

I don't know anything about Electron, but sounds like it should not be using ~/.config. If it's going to be a large dump of mixed-purpose files anyway, better move the directory elsewhere altogether (eg.~/.itch), although that would be just a workaround.

fasterthanlime commented 7 years ago

@shunsei just to clarify (my initial response had a rather bland tone): I completely sympathize with your request.

let's look at practical fixes:

However, there is an app.setPath method that can be called before the app is ready, that can adjust some paths. See this list.

So, it would be possible for the itch app to set different directories - but I'm not sure how to proceed in a backwards-compatible manner. It has been using ~/.config/itch (or $XDG_CONFIG_HOME/itch) for a while, and I can't implement a "migration assistant" within the app, since setPath needs to be called before the app is ready (so, before I can show any UI, for example), and doing it while the app is running isn't possible either (Chrome locks some files, so they can't be deleted/moved).

One possible way to do it would be:

Then, users could manually mv ~/.config/itch ~/.itch while the app isn't running. It might also be possible for the app to do itself on startup, synchronously (before the app has time to get ready), although if ~/.config and ~/.itch are on different partitions, it might fail (because then a rename operation won't work, only copying then deleting the original files would). That's a lot of things to do synchronously :/

Quick note on synchronicity

V8/Electron are basically big event machines. When starting up, it requires everything - when it does, each module owns the CPU - no other javascript code is executing at this time. Any synchronous operation (fs.writeFileSync for example) will retain control of the CPU. Asynchronous operations (fs.writeFile(function () { /* done! */ })) however, will relinquish control of the CPU, letting it execute something else.

It's going to be tricky to write the entire ~/.config/itch => ~/.itch migration routine entirely synchronously to make sure we own the CPU the whole time and the app doesn't have a change to get ready (which it does, once all required modules have given up controle of the CPU).

(It will also look like the app is taking forever to start, which is bad)

I hope this makes sense.

Another option would be to ship a native executable (I'd go with golang) which role would be to:

Lastly, ~/.itch was actually where I was thinking of having itchSetup install the itch app, so the final path may end up being ~/.itch/data instead.

I hope you understand why it's not done yet :)

Edit: a last solution comes to mind. If only tech-savvy users care about migrating ~/.config/itch to ~/.itch, the app could accept a --migrate-config argument, and behave as a CLI tool. If it doesn't open any windows, Chrome shouldn't get a chance to lock files and it wouldn't fail.

shunsei commented 7 years ago

Sounds pretty involved. From reading the Electron docs, appData really should not be pointing to $XDG_CONFIG_HOME, if it's touted as a generic "application data directory" (although userData does mention it's only for configuration).

I personally wouldn't mind having to move the directory manually, it's a one-time migration after all.

Would this new ~/.itch directory still contain all different kinds of files though?

fasterthanlime commented 7 years ago

Would this new ~/.itch directory still contain all different kinds of files though?

Yep, except if we go even more involved and start splitting folders even more.

My original design intention was that: chrome cache files should be small enough that it shouldn't matter where they're stored (I know it's not true for everyone 😞) - games/installed items are the main consumer of disk space, so you should get to pick where they're installed. Downloads also go wherever your main "install location" is.

fasterthanlime commented 6 years ago

itch v25 is out, and:

sparr commented 5 years ago

I am depressed to see that the decision was made to migrate from ~/.config/foo to ~/.foo instead of the appropriate ~/.local/share/foo and ~/.cache as suggested in the original issue report here.

fasterthanlime commented 5 years ago

I am depressed to see that the decision was made to migrate from ~/.config/foo to ~/.foo instead of the appropriate ~/.local/share/foo and ~/.cache as suggested in the original issue report here.

So, let's fix that.

What changed (between .deb/.rpm packages and itch-setup) was: the app's binaries moved from /usr/lib/itch to ~/.itch. Stuff that was in ~/.config/itch stayed where it was.

Now that itch-setup is a thing, and it works well, there is a path for me to migrate the ~/.itch stuff to ~/.local/share/itch and the ~/.config/itch to.. somewhere else, but probably not ~/.cache ? I don't know what a good default install location would be.

Anyway, please don't get depressed over that. I didn't just ignore all the feedback, there were a lot of tricky things to get right, and now that v25's been out for a while and is stable, we can:

I've just opened #2252 to track that.

fasterthanlime commented 5 years ago

Thinking about this some more, /cc @shunsei @sparr

Currently, itch v25 uses two directories:

So, with all that in mind, I'm not sure that I can map this cleanly to ~/.local/share/itch and ~/.cache

The only good candidate for ~/.cache (since that's supposed to be a directory) is.. GPUCache/. That's it. Everything else is not a pure cache (Partitions has game saves, db has installed game info, your login credentials, etc. - removing it would force you to re-log-in, re-scan for games, etc.)

As for ~/.local/share/itch, I've been going through mine to see what apps put in there, and.. I'm still not sure what it's for. I guess it's a better destination for "a mixed bunch of things" than ~/.config/itch.

Splitting up what currently goes in ~/.config/itch would be a headache - and by that I also mean "it could potentially blow up linux folks' install and take a while to resolve", which I really don't want. So, keeping all that in mind, how about:

Which leaves us with the following problems:

Would that be a step in the right direction or? I'm sort of debating what the advantages here would be compared to the amount of work it is.

eevee commented 5 years ago

Yeah, given that ~/.local/share is basically a junk drawer, I think the only Not Wrong option is to cram everything in there. If you can't break out config/cache, it's better to put config/cache in a more general place than to also put unrelated stuff in a more specific place.

paulloz commented 5 years ago

My two cents on the matter. I second the comment above.

Currently most of what's under ~/.config/itch/ would be better off in ~/.local/share/itch/. Keeping the actual configuration files under ~/.config/itch/ would be nice but if splitting up the folders are error prone for now, I'd rather see everything move out.
I personally don't like the idea of moving the content of ~/.itch/ to ~/.local/lib/itch/: itch and itch-setup doesn't belong there (at least). In addition I'm pretty sure this is not an XDG folder. As for ~/.cache/, how disk consuming is the GPUCache/ folder anyway? I'm not sure this is a huge issue at the moment.

fasterthanlime commented 5 years ago

GPUCache is less than one megabyte for me atm (but the app is not running). The larger caches are in Partitions, they're mostly images but mixed in with HTML5 save data / cookies

On Sat, Feb 23, 2019, 17:28 Paul Joannon notifications@github.com wrote:

My two cents on the matter. I second the comment above. Currently most of what's under ~/.config/itch/ would be better off in ~/.local/share/itch/. Keeping the actual configuration files under ~/.config/itch/ would be nice but if splitting up the folders are error prone for now, I'd rather see everything move out. I personally don't like the idea of moving the content of ~/.itch/ to ~/.local/lib/itch/: itch and itch-setup doesn't belong there. In addition I'm pretty sure this is not an XDG folder. As for ~/.cache/, how disk consuming is the GPUCache/ folder anyway? I'm not sure this is a huge issue at the moment.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/itchio/itch/issues/1184#issuecomment-466666505, or mute the thread https://github.com/notifications/unsubscribe-auth/AHoLZkb1px0IogWcV0nI2fg1FGOxqSgDks5vQWwegaJpZM4M3xRw .

fasterthanlime commented 5 years ago

So, moving ~/.config/itch to ~/.local/share/itch seems like a good (achievable) first step?

It would:

The process would be:

The only remaining file any of the tools would write to itch would be... ~/.config/itch/butler_creds, which is, well, actually used by butler to store your credentials. That doesn't really belong in config either, maybe it could move to ~/.local/share/butler/credentials ?

(Please vote :+1: or :-1: using comment reactions - if you vote against, please explain)

fasterthanlime commented 5 years ago

(note that, anywhere I say ~/.config, I mean $XDG_CONFIG_HOME falling back to $HOME/.config, and the same is true for ~/.local/share, ~/.cache etc.)

dos1 commented 5 years ago

butler_creds feels like could (or even should) stay in .config actually. Other than that, it's pretty much what I'd do myself ;)

tpruzina commented 5 years ago

What I expect to be in user directories (~user/<dir>/<appname>) goes along those lines: .config - user editable configuration (no database files, serialized crap or anything like that) .cache - cache files, user deletable in between sessions (e.g. your stuff needs to work whether or not this exists, is read writable, ...) .local/{share,lib,...}/ - random cruft, can be icons, database files, libraries or serialized data

Alternatively, start saving everything into ~user/.<appname>, and do whatever you want in there.

kuon commented 5 years ago

Just don't use ~/.itch (https://0x46.net/thoughts/2019/02/01/dotfile-madness/)

I'd put everything in $XDG_DATA_HOME. $XDG_CONFIG_HOME should only contain configuration that is intended to be edited "by hand". But if your config file is human readable and lives in the data folder, I don't case as long as I don't have to edit it.

For the cache dir, be sure to support its deletion, I delete mine regularly.

fasterthanlime commented 5 years ago

Just don't use ~/.itch (https://0x46.net/thoughts/2019/02/01/dotfile-madness/)

So, here's the thing that bothers me about this article: it says "just read the standard and follow it, duh!" - well, I've read the standard, a dozen times, and it doesn't say where to put applications that don't need root to be installed

I'm familiar with package manager guidelines that have strong opinions on where to put everything (and it differs by distro! Even after FHS!), and I'm still not convinced it's not a big ol' waste of time.

No, you don't need to edit any of the files itch drops on disk by hand, ever. You might want to force a clean install of the binaries, in which case you can just rm -rf ~/.itch - and your data is all still there. (The desktop shortcut that's install points to itch-setup --prefer-launch, which will repair your itch binaries if they're missing).

Again, ~/.local/lib and ~/.local/bin are not XDG standard directories. In fact, the whole ~/.local directory is a junk drawer. There's caches in there, you don't get rid of them by periodically running ~/.cache. Also, modern apps have cached data interleaved with other stuff for efficiency (you might want to use it in queries, it's less I/O operations, etc.)

How often do you actually ls -lhA ~ and what do you do it for? I'm still not convinced the entire reason behind these complaints is anything but ideological purity, which, uh, I don't have enough time for :(

kuon commented 5 years ago

I agree, the standard is not clear. Personally, I use arch and just create an AUR for all apps, except like 2 or three app images I use, I put those in ~/Apps/ but I didn't think about it, so I don't really know the best practice to install binaries in your home.

I'm against filling the home dir with folders because many file pickers display them, and it drives me crazy.

It's not your fault of course. But here is a save panel from firefox.

MestreLion commented 3 years ago

+1 for using ~/.local/share/itch for everything (since splitting is not easy/feasible)! It's a much much better "general purpose" area than ~/.config, and please never ever use ~/.itch!

Is this the current behavior? If so, please update https://itch.io/docs/itch/installing/linux/

By the way... there's absolutely nothing wrong using ~/.local/share/itch/bin to save executables. Sure, ~/.local/bin would be slightly better, (Python's pip already uses it), as it has the bonus of being added to $PATH if it exists in the default ~/.bashrc of many distros (including Debian/Ubuntu) since... 2016? But ok, it's merely a convention, not an official part of XDG spec.

XDG is not that complicated, if you think of its counterparts:

TL/DR: When in doubt, keep everything in XDG_DATA_HOME. It's a great compromise. It's far better than any other single-dir solution. And avoid using ~/.itchio at all costs, please!

ParaplegicRacehorse commented 2 years ago

+1 for use of XDG_DATA_HOME instead of XDG_CONFIG_HOME

Do not, under any circumstances, pollute my $HOME with a dot-directory which belongs in one of the XDG dirs.

MestreLion commented 2 years ago

@fasterthanlime

I'm still not convinced it's not a big ol' waste of time.

XDG is an attempt to create standards and guidelines on user's home, much like FHS does for the system. Splitting applications' data in several locations (/usr/share, /etc, /var) is common practice in any POSIX/Unix-like system, long before FHS, from Linux to BSD to MacOS. XDG did not invent this.

Again, ~/.local/lib and ~/.local/bin are not XDG standard directories.

They are not, but they're quite established conventions used by many software. And itch does not even need it: being a desktop application, you can set its .desktop launcher to point directly to its executable. It doesn't need to be in $PATH.

In fact, the whole ~/.local directory is a junk drawer.

No, it is not. Mine has only bin, lib and share subdirs. Most users have only share. And ~/.local/share has basically a subdir for each application, along with a few special subdirs: applications, icons, etc. That's much less junk than /usr/share.

There's caches in there

No there is not, unless inside a given application's ~/.local/share/myapp. And in that case I don't care if it makes its own subdir a junk drawer. Whatever goes inside ~/.local/share/myapp is considered implementation detail I don't need/have (or should) mess with.

How often do you actually ls -lhA ~ and what do you do it for? I'm still not convinced the entire reason behind these complaints is anything but ideological purity, which, uh, I don't have enough time for :(

It does make backups and system migrations a whole lot easier. Instead of special case every dotfile/dotdir of every app in my $HOME's root, I can simply backup/migrate ~/.config and all my settings are saved. I can choose to either save ~/.local or just let it re-create itself by re-installing stuff. And finally I can also delete ~/.cache anytime with no functionality loss.

MestreLion commented 2 years ago

Yes, software that existed long before XDG (or even FHS) still require special-casing, such as ~/.ssh or ~/.bashrc. That's not the case of itch, so we would really appreciate we didn't have to add yet-one-more-manually-included-exception to our already long backup scripts.

ChristianSilvermoon commented 2 years ago

I'm also bothered by this.

My current work around is to always execute Itch like so

HOME="$HOME/.local/opt/itch" ./itch

I currently use a wrapper script to do this for me and have my .desktop entry pointed at the wrapper script

Worth noting ~/.local/opt is also non-standard, but it makes sense to me as a user-level mimicry of the Hierarchy Standard, which is also kind of done with things like ~/.local/lib that a lot of other things do.

jrom99 commented 1 year ago

Is there any way to force itch to look for files inside .local/share insted of .config? Would it respect a symlink? Also, considering this is all done at setup, wouldn't it be better to move this issue to https://github.com/itchio/itch-setup/issues?

ChristianSilvermoon commented 1 year ago

Is there any way to force itch to look for files inside .local/share insted of .config? Would it respect a symlink? Also, considering this is all done at setup, wouldn't it be better to move this issue to https://github.com/itchio/itch-setup/issues?

You could consider copying my method and doing 

HOME="$HOME/.local/share/itch" ./itch

To force Itch to use that directory. It is worth noting that my solution comes with numerous (mostly benign) caveats that require creating Symbolic Links to your proper home directory to correct since this makes Itch treat the folder in question as your home directory.

At the very least I needed to symlink the following into the False Home Directory: Directory Reason
~/.themes User-side themes
~/.icons User-side icon/cursors
~/.local/share/icons User-side Icons/Cursors
~/.local/share/fonts User-side Fonts
~/.var Flatpak Support
~/.local/share/flatpak Flatpak Support

Most of these are optional, but I needed flatpak support for at least one title.

Use relative symlinks (AKA ../../../.themes and NOT ~/.themes) where possible, as migrating this setup to a different username will otherwise break your symlinks.

This is not ideal, but it's the best solution I know of.

jrom99 commented 1 year ago

This still doesn't solve the ~/.config/ situation, does it? I don't mind dotfiles on my $HOME, but a huge .config/ dir is another matter altogether.

ChristianSilvermoon commented 1 year ago

This still doesn't solve the ~/.config/ situation, does it? I don't mind dotfiles on my $HOME, but a huge .config/ dir is another matter altogether.

This workaround causes Itch to instead use ~/.local/share/itch/.config/itch rather than ~/.config/itch :P

It's not perfect, but it does prevent home directory clutter and keeps your Itch data out of your usual ~/.config and instead under ~/.local/share/itch

sparr commented 1 year ago

You could use unionfs to union mount your fake home on top of your real home and avoid needing the symlinks.