search

fer0n / Unwatched

RSS feed based video player for iOS
GNU General Public License v3.0
68 stars 7 forks source link

iCloud Sync. #7

Closed Revrean closed 3 weeks ago

Revrean commented 4 weeks ago

Hi,

Subscriptions, watched, inbox and queue don't seem to be syncing between my ipad 10.5' pro and iphone 15 pro. Could some kind soul please advise.

fer0n commented 4 weeks ago

Sync is working very unreliably currently and as far as I can tell the main issue is the amount of data it has to sync. I've been trying to reduce that. Could you follow these troubleshooting steps and see if that improves things for you?

General iCloud troubleshooting

App Specific Troubleshooting

Let me know how that goes and apologies for the inconvenience.

jchaps commented 4 weeks ago

So far, I have been able to get a subscription to sync (like, the fact that I’m subscribed to a certain feed). I even tried importing 51 subscriptions, and was able to get them all to sync.

I haven’t been able to get any actual video status to sync (e.g., which videos are in Inbox, and which are in queue). Obviously, this part feels like the whole point of sync to avoid having to go through the Inbox triage on each individual device.

I’m more than happy to keep testing and providing info if helpful.

fer0n commented 4 weeks ago

I'm assuming you're on the most recent version? I still think this is mainly a "sync is slow and needs a lot of time to catch up" issue.

There is no difference in sync between subscriptions and videos or queue/inbox entries. If you have 50 subscriptions and do a refresh, that's 50 subscriptions with 15 videos each = 750 videos, with a lot more data per model, plus 5 inbox entries (250) and for every chapter an additional model (10 chapters for every second video would be another 250). Its also not just syncing new models, its syncing changes, of which there are quite a few with each refresh. Additionally, it may have to catch up on any previously deleted models.

I'm not 100% sure if that's actually the issue, but I'm pretty sure it's at least part of it, especially when sync already works for the subscriptions. I've seen the initial sync take quite some time, subsequent ones are a lot faster.

Ideally, I'd only sync the queue and inbox and refresh the videos locally, the problem is you only get the 15 most recent videos per subscription, which means half of your queue could be empty rows on your second device. And I can't sync just some of the videos, it's all or nothing.

Perhaps I can find a way to further reduce the amount of things it has to sync, in the meantime my best suggestion is starting slow, refreshing with very few substitutions and giving it a lot of time to catch up.

jchaps commented 3 weeks ago

Yes, I’m on the most recent. I’ve been trying to troubleshoot and provide information about what’s going on in the hope that it helps you solve the issue, and each time I reinstall fresh from TestFlight.

The problem occurs with just one subscription:

  1. I did a full delete everything, delete app, reinstall
  2. I added one subscription, and saw the new sub sync to my other device. (Just the sub, not any videos)
  3. I refreshed the videos to add to the Inbox. These did not appear to sync my other device, even after several hours.
  4. I also tried adding a second subscription at this point. Sadly the second sub did not sync either (again after several hours of waiting).

In previous testing I was able to add several subs and they would all sync (including deleting subs synced) up until the first refresh of videos to the Inbox. Do you think that’s the main thing that sync is getting stuck on?

What is making the data model so large and slow? Is Unwatched downloading the actual video files, or do they stream from YT? Is there an API key or entitlement or something that you have that users don’t? Would it be possible to just sync the YT URL and status (inbox, queue, watched), and have the rest of the data about videos rebuild locally?

I will also offer that I have corporate MDM on my devices. I wouldn’t think that would cause an issue (it doesn’t for other apps I use, like Overcast), but I’ve seen MDM cause random bugs in the past so figured I’d mention it.

fer0n commented 3 weeks ago

In previous testing I was able to add several subs and they would all sync (including deleting subs synced) up until the first refresh of videos to the Inbox. Do you think that’s the main thing that sync is getting stuck on?

The initial refresh is creating a lot of new entries compared to only having the subscriptions, I do think that’s the main reason why it’s getting stuck.

Part of the problem could be that every time you set the app up again and restore from backup, everything that’s been there before is on still in the cloud so it might just get worse every time.

Perhaps this helps to “reset” CloudKit and actually start fresh: https://sowenjub.me/writes/how-to-delete-an-app-and-all-icloud-cloudkit-data/

What is making the data model so large and slow? Is Unwatched downloading the actual video files, or do they stream from YT?

In v1.0, the thumbnails were also being synced, which was unnecessary and made things a lot worse. I've since gotten rid of them and tried some other smaller improvements. No video files are being synced, it’s just meta data and urls. CloudKit seems to be an area of problems in general though and I frankly don’t know what exactly it’s doing under the hood.

Is there an API key or entitlement or something that you have that users don’t?

It’s working for some users, although many have issues with sync in general. Especially when parts are syncing, I don’t see anything like that being the issue. The only difference is the amount of data it’s syncing, at least to the best of my knowledge.

Would it be possible to just sync the YT URL and status (inbox, queue, watched), and have the rest of the data about videos rebuild locally?

If you want to, you can take a look at the data model here to see what values are being stored and synced. The short answer is I might be able to further reduce the stuff it has to sync by only syncing queue/inbox entries and keeping the videos on device, but it would come with some downsides: a lot more complexity, and not all videos would be on all devices (under a specific subscription and inside the "all videos" tab), probably some slower lists as well. I'm not sure if that's worth all that. Perhaps I can come up with a different solution.

jchaps commented 3 weeks ago

I did the full reset/delete across all devices.

I set up iCloud sync with just one sub on one device.

I left it alone for 24 hours.

Then sync started working great. I tried adding new subs, modifying videos in inbox and queue, and it all synced over immediately.

I was able to restore a backup with 50+ subs and 20+ videos in queue. And all that synced over in about 30 seconds.

I don’t know what exactly triggered the fix - if it was the full delete or the long wait - but it all seems to be working great now, and syncs quite quickly.

Revrean commented 3 weeks ago

Yeah I nuked (deleted backups, deleted app etc) everything and started fresh sync seems to work great now thanks for the help! Really appreciate it.

fer0n commented 3 weeks ago

Really happy to hear sync has finally started to work. There was also an issue in the CloudKit schema that I've fixed, it sounds like that may have been the main issue. Thanks for reporting back :)

fer0n commented 3 weeks ago

I frankly expected this to drag on for a very long time, but apparently that latest change has actually fixed it for everyone.