Mackup breaks on macOS Catalina DP1 when using iCloud (path change)

[fkyuga]

Prior to 10.15, the path for iCloud Drive was ~/Library/Mobile Documents/com~apple~CloudDocs. In 10.15, this path has changed to ~/Library/CloudStorage/iCloud Drive. Upgrading to Catalina causes old symlinks to break. Some sort of migration process will be necessary for these users in time for the release.

[jnbn]

For a temporary solution you can try listing them by

ls -la ~/ | grep "\->"

and after cd ~change them manually by:

ln -sf ~/Library/CloudStorage/iCloud\ Drive/Mackup/.zshrc .zshrc

[jnbn]

Thought a bash would be good but have no time to finish it. If there's anybody who can do it voluntarily it'd be like this much or less.

#! /bin/bash

DIR= "~"
OLD_PATTERN="/Users/jnbn/Library/Mobile Documents/com~apple~CloudDocs"
NEW_PATTERN= "/Users/jnbn/Library/CloudStorage/iCloud Drive/"

while read -r line
do
    echo $line
    CUR_LINK_PATH="$(readlink "$line")"
    NEW_LINK_PATH="$CUR_LINK_PATH"  
    NEW_LINK_PATH="${NEW_LINK_PATH/"$OLD_PATTERN"/"$NEW_PATTERN"}"
    rm "$line"
    ln -s "$NEW_LINK_PATH" "$line"
done <<< $(find "$DIR" -type l -maxdepth 1) 

[jnbn]

Seems like an it needs an update on

https://github.com/lra/mackup/blob/16cc85a4725c95db6ac6031f54223fe390fe5ee1/mackup/utils.py#L306

if 
defaults read loginwindow SystemVersionStampAsString > 10.15
yosemite_icloud_path = '~/Library/CloudStorage/iCloud Drive/

[joshmedeski]

The new macOS is called "Catalina". So it sounds like Mackup can be updated to detect Catalina and change the iCloud path dynamically.

[fearoffish]

Following on from what @jnbn said, I used the following:

#!/bin/bash

set -e

OLD_PATTERN="/Users/$USER/Library/Mobile Documents/com~apple~CloudDocs"
NEW_PATTERN="/Users/$USER/Library/CloudStorage/iCloud Drive"

for line in $(find . -type l -maxdepth 1)
do
    if [[ $(readlink $line) == *"CloudDocs"* ]]; then
        CUR_LINK_PATH="$(readlink "$line")"
        NEW_LINK_PATH="$CUR_LINK_PATH"
        NEW_LINK_PATH="${NEW_LINK_PATH/"$OLD_PATTERN"/$NEW_PATTERN}"
        ln -f -s "$NEW_LINK_PATH" "$line"
    fi
done

You have to run it in whatever folder has broken symlinks, it doesn't recursively search. Some examples:

/Users/$USER/Library/Preferences /Users/$USER/Library/Application Support /Users/$USER/

[noproxy]

The problem is not just simply repairing the iCloud path.

More seriously, reading files in iCloud now requires permissions since MacOS Catalina. And the permission must be granted for every programs who read preferences from iCloud directory. So I have to stop using iCloud as the backup engine.

[lra]

More seriously, reading files in iCloud now requires permissions since MacOS Catalina. And the permission must be granted for every programs who read preferences from iCloud directory. So I have to stop using iCloud as the backup engine.

Yes, looks like support for iCloud should be removed because I don't see a clean way to make it work.

[madsem]

More seriously, reading files in iCloud now requires permissions since MacOS Catalina. And the permission must be granted for every programs who read preferences from iCloud directory. So I have to stop using iCloud as the backup engine.

Yes, looks like support for iCloud should be removed because I don't see a clean way to make it work.

sighhh :( I just finished setting up my dotfiles and everything, using iCloud. I see what you're saying though. Man, Apple is killing me

[dstroot]

Yup, I can't use files/github because I would leak sensitive data (unless I use a private GitHub repo).

I removed dropbox from my machine

I love/loved iCloud. It was perfect. Not sure what I want to do now.

[madsem]

My solution was to write helper functions for my shell that use the mac ‘security’ command: https://ss64.com/osx/security.html

That way i can store secrets in the login keychain, and use command substitutes anywhere I use secrets. So files that are (now on Dropbox) saved don’t have plain text secrets.

I have it in my dotfiles: https://github.com/madsem/dotfiles/tree/master/.config/fish/functions

[fkyuga]

I decided to use Syncthing instead - it lets you easily sync files between your computers. I set it up on my home server and it works great.

[dstroot]

My solution was to write helper functions for my shell that use the mac ‘security’ command: https://ss64.com/osx/security.html

That way i can store secrets in the login keychain, and use command substitutes anywhere I use secrets. So files that are (now on Dropbox) saved don’t have plain text secrets.

I have it in my dotfiles: https://github.com/madsem/dotfiles/tree/master/.config/fish/functions

This is brilliant! Thanks for sharing!!

[josh1703658784]

Here's what I did to work around the issue.

Updated the normal file .mackup.cfg with:

[storage]
engine = file_system
path = Library/CloudStorage/iCloud Drive/Mackup
directory = <_your_chosen_directory_>
...

cp /path/to/normal/file/.mackup.cfg ~/.mackup.cfg

unlink "$(mackup -f restore 2>&1 >/dev/null | grep 'FileNotFoundError: \[Errno 2\] No such file or directory:' | cut -d"'" -f2)"; mackup -f restore;

You could probably throw it in "watch" to just keep running it until you stop seeing errors. Or a loop checking for return code if you wanna get fancy... I guess it depends how much stuff you have symlinked.

Basically this will unlink the old file that throws the error and then it'll run restore.

[scottrobertson]

Just upgraded to Catalina, every time i close apps such as Iterm2 and Bartender the settings revert. If i uninstall Mackup, it's fine. Seems to be an issue with ~/Library/Preferences using symlinks?

[lra]

I don't have Catalina. Can anyone confirm if linking preference files of an application in iCloud works?

[scottrobertson]

@lra not using engine = icloud, no.

It also breaks for me using file_system, and using a path inside iCloud.

[nicolinuxfr]

Same issue here. I guess mackup can not work anymore with macOS, starting with Catalina ?

[lra]

Same issue here. I guess mackup can not work anymore with macOS, starting with Catalina ?

Only when using iCloud, not the whole MacOS.

[nicolinuxfr]

Some comments seemed to indicate it was linked to the symbolic links, not iCloud.

Can someone confirm it works fine with Dropbox or Google Drive ?

[scottrobertson]

@nicolinuxfr it's odd. I have issues sometime with iTerm2 resetting its settings. I am using file_system sync, with git.

But most of the time, it's fine.

[nicolinuxfr]

@scottrobertson OK it's better than nothing, I guess.

I will try to set up Mackup without using iCloud Drive engine first, as suggested by @joshuaks.

[fkyuga]

I'm not on the Catalina beta anymore, but according to this tweet Apple has rolled back all iCloud changes in the latest betas of iOS 13 and Catalina. I may be wrong, but that might mean that Mackup is working again on the newest builds.

Can anyone confirm this?

[sviatoslav-lebediev]

@fzslm I think you are right, the last beta uses ~/Library/Mobile Documents/com~apple~CloudDocs, previously it was ~/Library/CloudStorage/iCloud Drive/ I've tried to back up, looks like everything is ok

[nicolinuxfr]

I can confirm, and what I did on beta 6 to make mackup works with Catalina means it’s all broken again with beta 7… 🤦‍♂️

I guess I will have to undo it somehow.

[jnbn]

I suggest using Google Drive or Dropbox. To change and move all backups to there; just change the config and makeup backup

Probably there won't be an easy solution for iCloud Drive soon.

[scottrobertson]

I switched to just using file_system, and then backing up to an encrypted Keybase Git Repo.

backup () {
    mackup backup && cd ~/Code/keybase/Mackup && git add . && git commit -am 'Backup' && git push && cd -
}

[nicolinuxfr]

I suggest using Google Drive or Dropbox. To change and move all backups to there; just change the config and makeup backup

Probably there won't be an easy solution for iCloud Drive soon.

OK but I don’t think it would work if current symlinks are broken… am I wrong ?

[MaxDiOrio]

Umm...isn't the simple solution this? It works flawlessly for me.

ln -s /Users/$USER/Library/Mobile\ Documents/com~apple~CloudDocs /Users/$USER/Library/CloudStorage/iCloud Drive

[kreeger]

Yeah, just to reiterate what was said here — I think the "path changes" at play were part of some larger iCloud Drive overhaul that Apple was working on for both iOS 13 and macOS 10.15 Catalina — changes that they've since rolled back across the platform because it wasn't ready in time for shipping. So the old paths remain where they were in 10.14 for now.

I'm betting they'll try and reintroduce iCloud Drive changes in 10.15.1 or 10.15.2, but I'm hoping they wait until 10.16. Anyway, this shouldn't be an issue for now, so this issue could probably be put on ice.

[tanpengsccd]

Following on from what @jnbn said, I used the following:

#!/bin/bash

set -e

OLD_PATTERN="/Users/$USER/Library/Mobile Documents/com~apple~CloudDocs"
NEW_PATTERN="/Users/$USER/Library/CloudStorage/iCloud Drive"

for line in $(find . -type l -maxdepth 1)
do
  if [[ $(readlink $line) == *"CloudDocs"* ]]; then
      CUR_LINK_PATH="$(readlink "$line")"
      NEW_LINK_PATH="$CUR_LINK_PATH"
      NEW_LINK_PATH="${NEW_LINK_PATH/"$OLD_PATTERN"/$NEW_PATTERN}"
      ln -f -s "$NEW_LINK_PATH" "$line"
  fi
done

You have to run it in whatever folder has broken symlinks, it doesn't recursively search. Some examples:

/Users/$USER/Library/Preferences /Users/$USER/Library/Application Support /Users/$USER/

I change maxdepth to 5 , it work for me to recursively search and change all links .

cd ~/

OLD_PATTERN="/Users/leo/Library/Mobile Documents/com~apple~CloudDocs"
NEW_PATTERN="/Users/tanpengsccd/Library/Mobile Documents/com~apple~CloudDocs"

for line in $(find . -type l -maxdepth 5)
do
    if [[ $(readlink $line) == *"CloudDocs"* ]]; then
        CUR_LINK_PATH="$(readlink "$line")"
        NEW_LINK_PATH="$CUR_LINK_PATH"
        NEW_LINK_PATH="${NEW_LINK_PATH/"$OLD_PATTERN"/$NEW_PATTERN}"
        ln -f -s "$NEW_LINK_PATH" "$line"
    fi
done