swyxio / swyxdotio

This is the repo for swyx's blog - Blog content is created in github issues, then posted on swyx.io as blog pages! Comment/watch to follow along my blog within GitHub
https://swyx.io
MIT License
336 stars 45 forks source link

What to do when iCloud is stuck on uploading items #437

Closed swyxio closed 1 year ago

swyxio commented 2 years ago

category: note

I use iCloud as my syncing engine for my Obsidian Second Brain, and twice now I've seen iCloud get corrupted into a really bad state. I also back up everything to github, so I dont really experience much data loss, but it is annoying to see iCloud be so unreliable at even the simple task of syncing mostly text files.

Today's issue was that I somehow created some files that iCloud didnt like (I renamed one file, and added a few others, in a short period of time, but nothing crazy), which led to something like this:

image

and iCloud then refusing to sync anything else that I added or modified to my synced folders from there on out (i was uploading 7kb, not 2gb, which made it even more infuriating that it was getting stuck uploading that trivial of data)

If you google this problem you get a lot of really horrible advice from the apple forums, like turn off your iCloud and on again and again (which deletes stuff that was just fine! if you have a large icloud account - i use mine for photos - that is ~100gb of stuff that just gets wiped and later has to be resynced), or just a depressing chain of black box user anecdotes that don't offer any confidence whatsoever.

I also tried killing and restarting Finder itself, to no avail.

I'll skip straight to the point - this is the advice that worked for me.

$ ps aux | grep bird

# output looks something like
swyx               597   1.7  0.1 408320592  38384   ??  S     7:19PM   0:26.45 /System/Library/PrivateFrameworks/CloudDocsDaemon.framework/Versions/A/Support/bird
# grab the process ID (PID) from this output - here it is 597

# optional: Check current priority of bird process
$ ps -fl -C [PID]
  UID   PID  PPID   C STIME   TTY           TIME CMD                     F PRI NI       SZ    RSS WCHAN     S             ADDR
  501   597     1   0  7:19PM ??         0:36.11 /System/Library/  1004004  31  0 408325200  44208 -      R                   0

# Change priority of the bird process
sudo renice -n -10 -p [PID]

once I did this, syncing resumed almost immediately.

swyxio commented 2 years ago

published it on my blog not because i think this has a wide audience, i literally just want to raise SEO ranking of the right answer

zxt-tzx commented 1 year ago

I face this problem too (exact same issue while using iCloud for Obsidian), but this solution doesn't work for me.

I used to run the following script (which essentially kills the bird process so as to restart it), but that has also stopped working for me a while ago.

swyxio commented 1 year ago

thanks for contributing!

zxt-tzx commented 1 year ago

OK I have done a more in-depth investigation of this issue and think I've found a solution — posting it here so that it might benefit anyone else who comes across your site (which seems to have decent SEO).

Scenario 1: Your iCloud syncing occasionally gets stuck, but not all the time.

Solution: Kill the bird process (the iCloud syncing service). You can kill it via Activity Monitor or use the shell script linked in my previous comment. If you find that this immediately brings the syncing the completion, then your problem is a mild one and you can stop reading.

Scenario 2: Your iCloud syncing gets stuck all the time (particularly when using Obsidian), and killing the bird process doesn't provide immediate help. In my experience, just because it gets stuck doesn't mean it never completes. It just takes syncs intermittently when, on a laptop with no bugs, the syncing is supposed to be immediate.

Solution: I have noticed that, per this comment, clicking the option to turn off your iCloud somehow bumps up the priority of iCloud syncing and can actually help the sync complete. However, this is not a real solution. This is because, after returning to regular usage, the syncing continues to be stuck all the time. If this is your experience, you should:

At this point, I'm not sure how necessary that last step is (will report back again once I've clarity on this), but so far my syncing is immediate and using Obsidian on my Mac, then on my phone, is exactly how it should be. All the best!

thatdebi commented 1 year ago

Hi, I was following the instructions above from @tan-zx but when I run the script I get: renice: 42296: setpriority: Permission denied renice: 601: setpriority: Permission denied renice: 854: setpriority: Permission denied renice: 4600: setpriority: Operation not permitted renice: 399: setpriority: Operation not permitted renice: 574: setpriority: Permission denied

Any advice? I've killed the bird and then restarted and it'll temporarily fix the problem, but as soon as I have anything new to sync it gets hung up again. I've also logged out entirely and restarted, and then I even reset the entire laptop and I'm still having this issue. :(

zxt-tzx commented 1 year ago

hi @thatdebi, really sorry to hear that. sudo is required to run that script (which is likely the cause of your error)

in my experience, I didn't find the script necessary to fix the problem. instead, what I found to have fixed the problem permanently was to sign out of Apple ID, restart, and sign back in again. thereafter, while iCloud sync might hang occasionally, killing bird with this script would solve it for a good period of time (i.e. approximately a week before it recurs)

hope this helps!

swyxio commented 1 year ago

thanks for being so nice in helping others @tan-zx :)

tophee commented 1 year ago

This issue is also driving me nuts. I have previously used killall bird to get the sync to complete, but when I tried it right now, it didn't help either. The upload re-started but didn't complete, just like before. @tan-zx, why are you using that script instead of killall bird? It looks as though some app is accessing a file onside the icloud folder and not letting it go, but lsof yields no file in any of the folders that have the cloud icon.

I also use Obsidian, so I closed down obsidian but the upload was still stuck. I suppose its not a coincidence that both os us with that problem are using obsidian. Also, the cloud symbol (indicating files that are not yet synced) is on my obsidian vault (and nothing else. But what is obsidian doing to break icloud sync?

zxt-tzx commented 1 year ago

have you tried this:

to sign out of Apple ID, restart, and sign back in again

tophee commented 1 year ago

That signing in and out will temporarily fix the issue is rather trivial. I don't even need to sign in and out, a restart is enough. But that doesn't fix the root of the problem. Hence my question: what is obsidian (or one of its plugins) doing that might break icloud sync?

I just found this: https://github.com/tgrosinger/recent-files-obsidian/issues/33

Are you using the recent files plugin? I am, so I will now disable it and see if that fixes the issue.

zxt-tzx commented 1 year ago

That signing in and out will temporarily fix the issue is rather trivial

fwiw it fixed the root of the problem for me (unlike a restart), i.e. it doesn't recur except very rarely, in which case killall bird would fix it

zxt-tzx commented 1 year ago

Are you using the recent files plugin? I am, so I will now disable it and see if that fixes the issue.

No, I'm not

solival commented 1 year ago

Thanks for the article. I've made oneliner to make things easier:

BIRDPID=`ps aux | grep -i bird | awk '{print $2}' | head -1`; sudo renice -n -
10 -p $BIRDPID
nikitapashinsky commented 1 year ago

Thank you for the article, and thanks to @tan-zx for their comment convincing me to do the whole sign out / restart / sign in process :-D Sync is lightning fast at the moment. Fingers crossed!

MicroTed commented 10 months ago

Renicing 'bird' didn't help me just now (it just sat at 0% CPU), but killing it did work (using Activity Monitor). Thanks!

lucky13820 commented 10 months ago

Somehow renicing doesn't work for me, bird process still has 31 as the priority. Killing bird didn't help as well, it will restart the uploading process and get stuck at some place.

cojestjagiello commented 3 months ago

It really helped me; the files started uploading to iCloud. I used all the commands mentioned in this article. I had an issue with the PID number. In my case, it was entirely different than expected, and I found it after a few attempts. I have almost no experience with coding, but if someone finds that this method doesn't work, they should try to find the actual PID number.

Screenshot of uploading
lucky13820 commented 3 months ago

So looks like the problem for me is that the PID for my bird process keeps changing, I can never set the priority because it won't get the right PID. Does anyone know a fix for this?

cojestjagiello commented 3 months ago

I went into "Activity Monitor," and there, after using the killall bird command with no success, I copied the PID of that process directly from Activity Monitor. The Terminal showed me a completely different PID than Activity Monitor. As I mentioned, I have almost no experience with such things, but in my case, using the PID from Activity Monitor worked.

The Second Screenshot today

However, after this whole process of trial and error, I'm so frustrated with iCloud and this situation that I just switched to Dropbox. It's worse than iCloud in some aspects, but hey, at least it lets me upload something without a fight (just kidding, sort of).