Open MicahZoltu opened 3 years ago
Thanks, @micahzoltu - we'll bring this up in our next weekly triage.
Did some spelunking on borrowed Windows 10 laptop and ($user
is the name of my user):
%USERPROFILE%
is C:/Windows/Users/$user/
%APPDATA%
is C:/Windows/Users/$user/AppData/Roaming/
%LOCALAPPDATA%
is C:/Windows/Users/$user/AppData/Local/
%PROGRAMDATA%
is C:\ProgramData
(?)
For sure there are historical reasons for this, but this naming convention is pretty awkward.
Anyway, IPFS Desktop creates:
C:/Windows/Users/$user/.ipfs/
- go-ipfs config and repoC:/Windows/Users/$user/.ipfs-desktop/
– config bridging GUI and CLIC:/Windows/Users/$user/AppData/Roaming/IPFS Desktop/
– Electron app config and UI cachesC:/Windows/Users/$user/AppData/Local/Programs/IPFS Desktop/
– Electron and go-ipfs binariesafaik same, with one difference:
C:/Program Files/IPFS Desktop/
– Electron and go-ipfs binariesiiuc the main issue here is with the first two, namely we have those ugly folders in %USERPROFILE%
directory:
My thoughts:
.ipfs
:
IPFS_PATH=$HOME/.ipfs
and on Windows $HOME
seems to be translated to %USERPROFILE%
IPFS_PATH
on Windows, when it is missing from env) and point at something like %LOCALAPPDATA%/go-ipfs/.ipfs
.ipfs-desktop
$XDG_CONFIG/IPFS Desktop
on Linux is $HOME/.config/IPFS Desktop
, and $user/AppData/Roaming/IPFS Desktop
on Windows)PATH
integration, nor customized binary/IPFS_PATH
, so risk is low@MicahZoltu does this sound sensible? Let me know if I missed some nuance.
I'm open for reviewing a PR if someone familiar with Windows creates one.
We could override the default (set custom
IPFS_PATH
on Windows, when it is missing from env) and point at something like%LOCALAPPDATA%/go-ipfs/.ipfs
A .
prefix on Windows does not alter visibility and it causes some mild annoyances in Windows Explorer since you cannot set a folder to that name in some contexts. However, applications are free to do whatever they want within their subfolders of %LOCALAPPDATA%
so if for some reason having a folder without a leading .
is difficult then letting it stay I think is fine and still respects Windows folder hierarchy.
We could safely remove this directory by moving it to the directory with Electron app config (
$XDG_CONFIG/IPFS Desktop
on Linux is$HOME/.config/IPFS Desktop
, and$user/AppData/Roaming/IPFS Desktop
on Windows)
$user/AppData/Roaming
should be %APPDATA%
. It is configurable, and won't always be a subdirectory of %USERPROFILE%
nor will it necessarily be a sibling of %LOCALAPPDATA%
(in fact, it is common to have them on different drives). Does IPFS currently use %APPDATA%
, %LOCALAPPDATA%
, and %USERPROFILE%
or is it manually constructing those paths? If it is manually constructing those paths then that should be fixed as well.
Something to be aware of is that Electron's default locations are bad and you should not assume that Electron will put data in the right place by default. Electron by default considers hundreds of megabytes of cache data to be "app config" and it stores it in a location that is network synced/backed up. Given this, I recommend moving the Electron stuff to %LOCALAPPDATA%
since it can grow pretty large and is transient data (not actual user-configuration data).
In general, my advice is to use %LOCALAPPDATA%
for everything as a default unless your application is specifically designed to support roaming/network synced configuration. I don't know if IPFS is currently designed for synced user configuration, though it would be pretty cool if it was. In particular, I would like it if my list of pinned files was synced and when I switched computers IPFS would automatically fetch pinned items and remove pins from items I unpinned on another computer. However, I would not want the actual pinned content to live in a roaming location, just the hashes.
Thank you @MicahZoltu, that is very useful feedback.
Did some digging and there are three separate topics here, each can be tackled separately:
%APPDATA%
with %LOCALAPPDATA%
Main concern here is the cost of fixing Electron's shortcomings in our code over time.
We tried doing that many times in the past, and then when electron
or electron-builder
finally fixed the problem in upstream code, it broke our workarounds in terrible ways.
My opinionated advice here is to wait for upstream electron
to fix the %APPDATA%
vs %LOCALAPPDATA%
problem.
Personally, I would not pick this up, as it is feels like a huge rabbit hole without any visible improvement to users, but if someone wants to tackle this, I'd be open to reviewing a PR that fixes paths for new Windows installations, as long it:
electron
changes their defaults at some pointIPFS Desktop/
exists in %APPDATA%
, use it instead of creating new one in %LOCALAPPDATA%
.ipfs-desktop
from %USERPROFILE%
to Electron app config dirThis can be tackled as a standalone PR:
$HOME/.ipfs-desktop/*
to Electron app config dir (on Linux $XDG_CONFIG/IPFS Desktop/
)
.ipfs
from %USERPROFILE%
to %LOCALAPPDATA%
The %USERPROFILE%
is the default picked by go-ipfs
daemon when IPFS_PATH
is missing.
Code responsible for this is in https://github.com/ipfs/go-ipfs/blob/v0.7.0/repo/fsrepo/misc.go#L13-L18 and https://github.com/mitchellh/go-homedir/blob/af06845cf3004701891bf4fdb884bfe4920b3727/homedir.go#L148-L166
I think ipfs-desktop could avoid polluting %USERPROFILE%
if we:
IS_WIN
)
IPFS_PATH
is set, use it as-is%USERPROFILE%/.ipfs
exists, use it as IPFS_PATH
to ensure existing installations do not breakIPFS_PATH
to be something explicit like %LOCALAPPDATA%/go-ipfs/IPFS_PATH
, so new installations on Windows won't pollute %USERPROFILE%/
ps. The problem will remain if user runs go-ipfs without Electron wrapper. The default IPFS_PATH
is a decision go-ipfs daemon does, so perhaps this should be also fixed upstream in BestKnownPath
too.
I submitted an issue to get go-homedir
library fixed: https://github.com/mitchellh/go-homedir/issues/30
I believe there are other similar libraries that better support different data types so if the author of go-homedir
is resistant to this change, perhaps switching go-ipfs
to a different library would be the right course of action long term. Either way, migration will need to be handled still so the library getting fixed isn't the final solution.
Main concern here is the cost of fixing Electron's shortcomings in our code over time.
My opinionated advice here is to wait for upstream electron to fix the %APPDATA% vs %LOCALAPPDATA% problem.
My concern with this strategy is that this issue has been open with Electron for almost 4 years now, and until they fix it the default of Electron is to be a bad citizen. This means that IPFS Desktop will continue to be a bad citizen indefinitely, since there is no light at the end of the tunnel for Electron fixing the issue.
That being said, I understand your position and appreciate you clearly laying out why you are weakly opposed to it. I certainly don't envy the maintainers of IPFS Desktop being put in this situation.
All in all, I agree with all 3 of your suggested paths forward. They are prudent and get things to a better place eventually without risking too much code complexity or breakage along the way.
As an intermediate step, can we get a configuration option added that will allow me to tell IPFS Desktop where to write? It seems that Electron has no intention of fixing this (going on 5 years open bug with no activity) and IPFS is probably the worst offender if you actively use it and pin with it (I'm looking at 2GB of "cache data" in my .ipfs folder right now).
Note: I tried setting %USERPROFILE%\.ipfs-desktop\IPFS_PATH
to point somewhere else and I also tried setting an environment variable for IPFS_PATH. In the first case, it just overwrote my IPFS_PATH
file on startup. In the second case, it didn't do anything.
It seems that after first launch the path is stored in %APPDATA%/IPFS Desktop/config.json
and reconstituted from there. Unfortunately, there doesn't appear to be any way to tell it to not write the .ipfs-desktop
folder to %USERPROFILE%.
Thank you @MicahZoltu, this is useful feedback. IIUC someone familiar with Windows may be able to fix this for new users with a set of surgical tweaks (only on Windows platform):
IPFS_PATH
is NOT set, and proactively set it to a better location in env with which go-ipfs binary is called – this will solve the .ipfs
placement in a way that won't break in the future.ipfs-desktop
– I believe on Windows we only use it in:
./src/daemon/index.js
./src/daemon/daemon.js
./src/ipfs-on-path/scripts/bin-win/ipfs.cmd
(deprecated experiment, but some users still run it)We don't have bandwidth for doing going this extra mile on top of Electron defaults, but if someone really cares, PRs are welcome :pray:
I suspect that Linux users would also like to see this fixed to respect XDG_CONFIG paths, and I think OSX similarly has some feature for defining where cache files go. The solution will likely be a bit different for each platform, so it does seem like fixing each separately may make the most sense (especially if you are looking to outsource the work).
Extracting action plan for most valuable fix for Windows users (https://github.com/ipfs/ipfs-desktop/issues/1862):
If a user installs IPFS Desktop on Windows, it will create multiple folders:
%AppData%\Roaming\IPFS Desktop %USERPROFILE%.ipfs %USERPROFILE%.ipfs-desktop
That's pretty unintuitive where what data will be stored and why.
Describe the solution you'd like Let IPFS Desktop create a single folder in %AppData% and move the rest of the folders below it.
Only caveat here is to do some magic similar to https://github.com/ipfs/ipfs-desktop/issues/1656#issuecomment-832295629 (need someone to check if that still makes sense) to ensure change applies to new installs and does not break repos of existing users.
Describe the bug Installing IPFS Desktop on Windows results in folders in %USERPROFILE% as well as cache and DB files in the roaming profile.
To Reproduce Steps to reproduce the behavior:
.ipfs
and.ipfs-desktop
folder in%USERPROFILE%
and%APPDATA/IPFS Desktop
contains cache files, databases, blob storage, etc.Expected behavior Cache files, file storage, databases, and per-machine configuration should all be stored in
%LOCALAPPDATA%/ipfs/
. Per user configuration (I'm not sure if IPFS has any of this, maybe the list of pinned files?) should be stored in%APPDATA%/ipfs/
. Nothing should be stored in%USERPROFILE%
(aka:%HOME%
) unless the user *explicitly tells the app to do so such as via a save dialog box.Additional context Both Linux and OSX also have environment variables that instruct applications where the appropriate place for files is. I believe it is called XDG_CONFIG though I'm not personally familiar with it.