Rebasing and file syncing

[cflorion]

I am using CDEMultipeerCloudFileSystem. MultipeerConnectivity takes care of syncing files between iOS devices. From what I understand when there is two many files on a device, Ensembles automatically does a rebase. Which means files are deleted and a new baseline is created.

Is rebasing deterministic ? I mean does it happen at the exact same moment (meaning with the same files) on every devices ?

Let's imagine 2 devices do a rebase not at the exact same moment. After files deletion, do these 2 devices will be fine is they sync each other files ?

(i'm asking because I have a syncing problem, and I thought this might be the cause)

Thanks ;)

[drewmccormack]

It is possible that two devices generate different baselines, but they should end up being deterministically consolidated into a single baseline, and all devices should end up adopting the same baseline.

What issue do you see? Maybe I can predict what might be happening to cause it.

Drew

[cflorion]

Hmm I see. The problem I have is that during a rebase, the changes I make on a device take really long to appear on the others. But you're right, after a few syncs the peers end up having the same baseline. This takes long, I think, because the rebase is REALLY done when all peers has done a rebase. This means that when peerA just did a rebase, the other peers think peerA is missing files (the ones it just deleted) so the other peers will send peerA the missing files. But after all peers did a rebase, everything goes back to normal.

Would it be a good idea for peerA to tell the others "i did a rebase" so they do the same ? That would speed up the process.

[drewmccormack]

Yeah, I imagine it is something like what you say. Eg.

The two grab new files from each other They merge, and this triggers a rebase on both. They probably do the same rebase. They export the new baselines, one each. Next time they sync, they get each others baselines, import them, and then delete one (deterministically).

This is all a waste of time in this case. It can happen with cloud syncing too, but is less likely.

The good news it is all robust, in the sense that they will eventually be back in sync. Bad news is there is a lot of time and energy wasted.

We should think carefully about a good way to deal with this. A simple solution may be to introduce a stochastic element to rebasing. Rather than definitely rebasing, it only rebases with some probability. It might not even be stochastic, but a relatively minor variation of the threshold based on the device unique id.

I think something like this would probably work well enough, and be very easy to do, as opposed to some sort of explicit communication, with devices having to wait for each other or something like that.

I'll think about it...

Drew

On Sep 4, 2014, at 15:18, Florion notifications@github.com wrote:

Hmm I see. The problem I have is that during a rebase, the changes I make on a device take really long to appear on the others. But you're right, after a few syncs the peers end up having the same baseline. This takes long, I think, because the rebase is REALLY done when all peers has done a rebase. This means that when peerA just did a rebase, the other peers think peerA is missing files (the ones it just deleted) so the other peers will send peerA the missing files. But after all peers did a rebase, everything goes back to normal.

Would it be a good idea for peerA to tell the others "i did a rebase" so they do the same ? That would speed up the process.

— Reply to this email directly or view it on GitHub.