App settings will not persist on macOS 14

scottrobertson commented 1 year ago

I have been trying to work out why my app settings are all resetting, and not persisting in macOS 14. This has been happening for apps such as Raycast, iTerm, Rectangle, Bartender etc. I just tried running mackup uninstall and everything is fixed. And then when i run mackup backup again, none of my settings will persist or save again. Any ideas?

joshmedeski commented 1 year ago

Which files storage method are you using? Can you share your ~/.mackup.cfg with us?

scottrobertson commented 1 year ago

[storage]
engine = icloud

[applications_to_ignore]
oh-my-zsh
espanso

iCloud

joshmedeski commented 1 year ago

I've seen other mackup users using iCloud have has similar issues. Could you try turning off the feature to optimize mac storage and see if that helps? It will be under the iCloud settings.

My thinking is iCloud is removing these config files from your hard drive to save storage.

scottrobertson commented 1 year ago

Hey. Just tried that, and it did not help, sadly.

I also did the following:

mackup uninstall
Open an app, all settings are fine
Rename/remove ~/Library/Mobile Documents/com~apple~CloudDocs/Mackup
Run mackup backup
Reopen the same app, and it has reset all settings
mackup uninstall
Reopen the app, and all the settings are back

joshmedeski commented 1 year ago

Can you confirm the symbolic links that are generated are valid? The Rename/remove step you mentioned could be causing the symbolic links to break.

@lra any ideas?

scottrobertson commented 1 year ago

To be clear, I had already run mackup uninstall before the rename/remove. I ran that earlier today to get things working. I did the rename/remove just to basically do a "fresh start" to rule out any old files causing issues etc.

scottrobertson commented 1 year ago

Hey. Just an update here. I just tried this again using

[storage]
engine = file_system
path = Mackup/

And i get the same result. I run mackup backup, open iTerm2 for example, and all settings have wiped, run mackup uninstall and open it again, and settings are back. So i don't think iCloud is the issue.

codechenx commented 1 year ago

I have same issue

ahmadawais commented 1 year ago

Same issue. Started happening when I updated to Dropbox beta, and then tried moving to iCloud. Now it's a mess. Not sure what's wrong, unable to predict a pattern here but with latest macOS this happens:

All of a sudden some apps loose their config (have to reconfigure)
Some apps like iTerm2 or Terminal don't keep their configs no matter what I do (uninstall/install fresh doesn't help)

Skowt commented 1 year ago

+1 on this issue. I was going to say I wonder if this is related to the Apple update that basically made it harder for Cloud apps to work on mac. It also meant you had to change the location of your Dropbox drive.

https://help.dropbox.com/installs/macos-support-for-expected-changes

scottrobertson commented 1 year ago

+1 on this issue. I was going to say I wonder if this is related to the Apple update that basically made it harder for Cloud apps to work on mac. It also meant you had to change the location of your Dropbox drive.

https://help.dropbox.com/installs/macos-support-for-expected-changes

I don’t think it’s that, no. It even stops working if I just use the file system storage method.

Skowt commented 1 year ago

Then I would say it's a recent Apple update. For me this started happening in the last week, which is when I updated to Ventura 13.4.1 (might've also been when I updated to 13.4).

It's possible that there's an unknown bug with the latest update or a "security improvement" that's messing with symlinks 🤔

Skowt commented 1 year ago

I've done mackup uninstall for now which has fixed everything but it would be interesting if mackup has any logs stored somewhere? To see exactly what was erroring out. Perhaps a permissions issue that was going undetected? Who knows at this point.

luispedrofonseca commented 1 year ago

Just lost a couple days on this exact issue because it took me a while to figure why a lot of my settings weren't saving and that mackup was the culprit. Hope this gets solved quickly.

jagajaga commented 1 year ago

cc me

jacobgraf commented 1 year ago

Any updates on this one? I am having the same issues and it's maddening. I will uninstall for now, but hope it gets fixed soon because I really love Mackup.

MacOS: 14.0 Public Beta (23A5286i) Mackup: 0.8.37

[storage]
engine = file_system
path = SynologyDrive/Backups

[applications_to_ignore]
illustrator
mackup
zsh
path-finder
streamdeck

davdroman commented 1 year ago

If for some reason this issue is due to symlinks and effectively impossible to fix going forward, it might be a good idea to offer an alternate mode of backup by copying over files instead of symlinking them, so that at least the tool doesn't become fully obsolete. I wouldn't mind doing a manual export/import every once in a while as a compromise instead of ditching Mackup altogether.

ahmadawais commented 1 year ago

I'm quite frustrated with losing all the settings again and again. To a point where I'm planning on rewriting this project one way or the other. It's been mentally taxing. 😮‍💨

I understand life gets in a way and this free software. Appreciate all the hardwork.

jacobgraf commented 1 year ago

If for some reason this issue is due to symlinks and effectively impossible to fix going forward, it might be a good idea to offer an alternate mode of backup by copying over files instead of symlinking them, so that at least the tool doesn't become fully obsolete. I wouldn't mind doing a manual export/import every once in a while as a compromise instead of ditching Mackup altogether.

Exactly. Honestly, I wish this was a feature today. I don’t really need the symlinking anyway. I just want the peace of mind that when I run mackup backup, my settings are copied somewhere that is versioned and backed up.

godbout commented 1 year ago

I've seen other mackup users using iCloud have has similar issues. Could you try turning off the feature to optimize mac storage and see if that helps? It will be under the iCloud settings.

My thinking is iCloud is removing these config files from your hard drive to save storage.

same with the new Dropbox in CloudStorage.

jacobgraf commented 1 year ago

I've seen other mackup users using iCloud have has similar issues. Could you try turning off the feature to optimize mac storage and see if that helps? It will be under the iCloud settings.

My thinking is iCloud is removing these config files from your hard drive to save storage.

same with the new Dropbox in CloudStorage.

I don't use either of these. I use SynologyDrive and have it set to sync everything. I don't think it has anything to do with the storage engine. I've had to temporarily remove Mackup to get things working, but like I said, I'm hoping they add a way to "copy" your prefs to another location instead of doing the whole symlink thing. That would be fine and preferable, for me.

godbout commented 1 year ago

for me things started failing with Dropbox when they moved to the new CloudStorage shit. (see https://github.com/lra/mackup/issues/1891.) now you can't be certain that Dropbox starts at a proper time. i think the whole symlink thing should be gone yeah. maybe there's something also related to macOS 14 itself, where now it seems we need Apple's approval to read and write our own UserDefaults 🤣️

neoromantic commented 1 year ago

I've been debugging same issue for past hour, and finally understood that it's due to mackup.

I'm running Sonoma beta.

Here's how it look:

rejecting write of key(s) mainWindowPositionCache in { com.raycast.macos, sergey.petrov, kCFPreferencesAnyHost, /Users/sergey.petrov/Library/Preferences/com.raycast.macos.plist, managed: 0 } from process 7061 (Raycast) because Path not accessible

romancin commented 1 year ago

Same problem here with Sonoma RC. Mackup is broken :(

iarchean commented 1 year ago

I am experiencing a similar issue, but it is quite strange. My Intel iMac is functioning properly, while my M2 MacBook has stopped working.

olssonm commented 1 year ago

Can confirm, having major issues in Sonoma, both for app settings but also system plists; (com.apple.* for example). Discovered it when settings for Tower (the Git-client) and system-level keyboard shortcuts (Library/Preferences/com.apple.symbolichotkeys.plist) didn't persist between reboots.

mackup.cfg:

[storage]
engine = icloud

[applications_to_ignore]
zsh

gabrielkoerich commented 1 year ago

Same issue, settings are not persisted.

Only 'fix' for now is mackup uninstall

my backup workflow is now mackup backup --force && mackup uninstall

reitermarkus commented 1 year ago

Just as additional information, defaults also doesn't work if the preferences file is a symlink but works with a plain file. So definitely symlinks are the problem.

ChrisChinchilla commented 1 year ago

Same here as soon as I installed Sonoma, damn… I have a lot of other "non-standard" setup with symlinks too, I really hope they're not all broken…

Edit… They still seem to work thankfully.

HBSpy commented 1 year ago

Same problem when upgraded to Sonoma, really want a copy option or use FSEvents API to auto copy config files

Apps use plist are broken, eg iTerm2, Apple Music, others like vim are fine

lucasmouilleron commented 1 year ago

Jendker commented 1 year ago

If anyone with MacOS 13.6 or 12.7 could confirm then it could be it.

lucasmouilleron commented 1 year ago

My temp workflow: Saving configs (when leaving a machine): mackup backup && mackup uninstall Loading configs (on new machine): mackup restore && mackup uninstall Mackup sync feature is bypassed but still better than nothing as mackup still locates/copies files.

davdroman commented 1 year ago

Same for me, except I skip prompts:

$ mackup backup --force && mackup uninstall --force

$ mackup restore --force && mackup uninstall --force

1activegeek commented 1 year ago

Going to just chime in for subbing that I'm also seeing this issue. Started when my Zoom global keyboard shortcut somehow kept resetting after a reboot. Turns out that is synced with mackup, and found the same symlink issue. Will take this workaround for now to backup/restore as mentioned. Hoping someone can figure out what Apple did.

m0rg5 commented 1 year ago

I'm on Ventura 13.6, 2020 m1, and just tried to reinstall mackup after I already brew uninstall mackup (as that didn't remove the symlinks) and eventually tracking the issue that I had with iina not saving prefs to here (ref'd above).

EDIT: seemed to get to uninstall after much finnaggling (still shows a dropbox not installed error when dropbox was neither confgured nor listed in mackup.cfg, go figure), still to test if prefs are saving in various apps properly. This is a helluva mess!

So, I brew install mackup again (along with crap-ton of dependencies) so I could try mackup uninstall, which I hoped would remove symlinks (which I didn't even realise Mackup was doing, should have RTFM) but instead got:

mackup uninstall
/opt/homebrew/Cellar/mackup/0.8.37/libexec/lib/python3.12/site-packages/docopt.py:165: SyntaxWarning: invalid escape sequence '\S'
  name = re.findall('(<\S*?>)', source)[0]
/opt/homebrew/Cellar/mackup/0.8.37/libexec/lib/python3.12/site-packages/docopt.py:166: SyntaxWarning: invalid escape sequence '\['
  value = re.findall('\[default: (.*)\]', source, flags=re.I)
/opt/homebrew/Cellar/mackup/0.8.37/libexec/lib/python3.12/site-packages/docopt.py:207: SyntaxWarning: invalid escape sequence '\['
  matched = re.findall('\[default: (.*)\]', description, flags=re.I)
/opt/homebrew/Cellar/mackup/0.8.37/libexec/lib/python3.12/site-packages/docopt.py:456: SyntaxWarning: invalid escape sequence '\S'
  split = re.split('\n *(<\S+?>|-\S+?)', doc)[1:]
Traceback (most recent call last):
  File "/opt/homebrew/bin/mackup", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/homebrew/Cellar/mackup/0.8.37/libexec/lib/python3.12/site-packages/mackup/main.py", line 66, in main
    mckp = Mackup()
           ^^^^^^^^
  File "/opt/homebrew/Cellar/mackup/0.8.37/libexec/lib/python3.12/site-packages/mackup/mackup.py", line 24, in __init__
    self._config = config.Config()
                   ^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/mackup/0.8.37/libexec/lib/python3.12/site-packages/mackup/config.py", line 46, in __init__
    self._parser = self._setup_parser(filename)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/mackup/0.8.37/libexec/lib/python3.12/site-packages/mackup/config.py", line 150, in _setup_parser
    parser = configparser.SafeConfigParser(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'configparser' has no attribute 'SafeConfigParser'. Did you mean: 'RawConfigParser'?

Doesn't bode well.

Also tried:

mackup backup --force && mackup uninstall --force
/opt/homebrew/Cellar/mackup/0.8.37/libexec/lib/python3.12/site-packages/docopt.py:165: SyntaxWarning: invalid escape sequence '\S'
  name = re.findall('(<\S*?>)', source)[0]
/opt/homebrew/Cellar/mackup/0.8.37/libexec/lib/python3.12/site-packages/docopt.py:166: SyntaxWarning: invalid escape sequence '\['
  value = re.findall('\[default: (.*)\]', source, flags=re.I)
/opt/homebrew/Cellar/mackup/0.8.37/libexec/lib/python3.12/site-packages/docopt.py:207: SyntaxWarning: invalid escape sequence '\['
  matched = re.findall('\[default: (.*)\]', description, flags=re.I)
/opt/homebrew/Cellar/mackup/0.8.37/libexec/lib/python3.12/site-packages/docopt.py:456: SyntaxWarning: invalid escape sequence '\S'
  split = re.split('\n *(<\S+?>|-\S+?)', doc)[1:]
Traceback (most recent call last):
  File "/opt/homebrew/bin/mackup", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/homebrew/Cellar/mackup/0.8.37/libexec/lib/python3.12/site-packages/mackup/main.py", line 66, in main
    mckp = Mackup()
           ^^^^^^^^
  File "/opt/homebrew/Cellar/mackup/0.8.37/libexec/lib/python3.12/site-packages/mackup/mackup.py", line 24, in __init__
    self._config = config.Config()
                   ^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/mackup/0.8.37/libexec/lib/python3.12/site-packages/mackup/config.py", line 46, in __init__
    self._parser = self._setup_parser(filename)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/mackup/0.8.37/libexec/lib/python3.12/site-packages/mackup/config.py", line 150, in _setup_parser
    parser = configparser.SafeConfigParser(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'configparser' has no attribute 'SafeConfigParser'. Did you mean: 'RawConfigParser'?

1activegeek commented 1 year ago

I'm on Ventura 13.6, 2020 m1, and just tried to reinstall mackup after I already brew uninstall mackup (as that didn't remove the symlinks) and eventually tracking the issue that I had with iina not saving prefs to here (ref'd above).

EDIT: seemed to get to uninstall after much finnaggling (still shows a dropbox not installed error when dropbox was neither confgured nor listed in mackup.cfg, go figure), still to test if prefs are saving in various apps properly. This is a helluva mess!

So, I brew install mackup again (along with crap-ton of dependencies) so I could try mackup uninstall, which I hoped would remove symlinks (which I didn't even realise Mackup was doing, should have RTFM) but instead got:

https://github.com/lra/mackup/issues/1948 - this find helped me fix this issue. Seems widespread. I made the change myself manually in config.py and appsdb.py per the linked PR, you can see what needs to change. This will help anyone having similar error of: AttributeError: module 'configparser' has no attribute 'SafeConfigParser'. Did you mean: 'RawConfigParser'?

ChrisChinchilla commented 1 year ago

Does the latest version fix this? It seems to fix other issues, but iTerm specifically still has the same error

castus commented 1 year ago

Doesn't fix it for me. It seems like new macOS doesn't follow symbolic links in Preferences. I think that the only fix is to replace symbolic links with the files.

ashtonian commented 1 year ago

I think this is still able to set symbolic links in Sonoma https://github.com/nickzman/symboliclinker/blob/master/SymbolicLinker.m

ChrisChinchilla commented 1 year ago

I thought it was (apart from iTerm) and then I restarted and everything reset, so uninstalled again for now :(

castus commented 12 months ago

I think the only thing we can do is to change the mackup behaviour from symlinking to copying files to the backup dir. @lra what do you think?

jacobgraf commented 12 months ago

I think the only thing we can do is to change the mackup behaviour from symlinking to copying files to the backup dir. @lra what do you think?

Personally, I’d love this option anyway. I don’t necessarily need symlinks. I just want to copy all my prefs to another folder that’s backed up and revisioned.

olimorris commented 12 months ago

I think the only thing we can do is to change the mackup behaviour from symlinking to copying files to the backup dir. @lra what do you think?

I think this is the way. Only concern is this could be an unexpected change for a lot of users who don't read the repo and for users who have symlinks currently in place.

castus commented 12 months ago

I think the only thing we can do is to change the mackup behaviour from symlinking to copying files to the backup dir. @lra what do you think?

I think this is the way. Only concern is this could be an unexpected change for a lot of users who don't read the repo and for users who have symlinks currently in place.

We can either change a major version number, to indicate a breaking change or fork it to the new repo that will have only copying feature. Migrating from symlink to copy will be easy as mackup restore && mackup uninstall.

I'm wonder what @lra thinks as he is the author ;)

ChrisChinchilla commented 12 months ago

I'm unsure if this is related, but I'm also finding that now even though I have uninstalled mackup completely, certain applications keep losing their preferences on quit, I am not even sure if they write a preferences file. I can see console is full of prefsd errors and I am not really sure what to do about this one and it's getting quite annoying 😅

austenc commented 11 months ago

@ChrisChinchilla I'm experiencing the same thing.

Has anyone found a fix for that? Using mackup uninstall removes some symlinks but many applications now cannot save their preferences.

iTerm BetterTouchTool F.lux VS Code

Yeah, it seems latest versions of mac don't like symlinking .plist files.

EDIT: For anyone who finds this -- here's what worked for me to recover some stuff:

mackup uninstall didn't remove all the symlinks that I had. Some still stuck around.
The above applications have .plist files stored in directories like ~/Library/Preferences/com.microsoft.vscode or similar folders. So in terminal you can find all your apps like this by doing cd ~/Library/Preferences && ls -la -- you will see the symlinks. You might check your ~/Library/Application Support/ folder too (escape the space in the terminal).
I replaced those symlinks with the actual folders and those apps will now persist settings correctly again.
Replacing all symlinks in both those folders with the original config files (still stored in Dropbox thankfully) fixed it for me

Will be looking to another symlink-free backup solution. 🥵

halilim commented 11 months ago

~~As a temporary measure, I replaced all plist soft links with hard links:~~

for file in $MACKUP_DIR/**/*.plist; do
  ln -f "$file" "~/$file"
done

~~They are still backed up~~
~~mackup backup will consider these new, and ask to soft-link again - say "No"~~

Update: It seems the hard links get overwritten with the actual files, so this method is not reliable.

Note that I also maintain a Git repository with a separate remote in the Mackup folder, so I can revert if everything goes haywire.

castus commented 11 months ago

I don't think that there is any other solution to this problem rather than invent a backup mechanism with a different behaviour than symlinking. For MVP it could be a one-way copy. There is even PR for that https://github.com/lra/mackup/pull/1947 It would be good if @lra speaks up and help us fix this issue.

ChrisChinchilla commented 11 months ago

Yeah, I ended up just manually replacing everything.

lra / mackup

App settings will not persist on macOS 14 #1924