Allow for onboarding using EBT in case partial replication is not supported

[KyleMaas]

This makes it so you can onboard against clients which do not support partial replication, such as Patchwork, by doing the initial onboarding using EBT. Fixes #25.

Also, don't use the global SSB object when not needed.

(If you want to make changes to this, you should be able to. I'd file this as a branch in your repo but I don't have permissions on core.)

[KyleMaas]

I figure this will be needed not just for rooms but also for direct connections with other client programs over Bluetooth or DHT.

[arj03]

I'm not sure if it's a good idea to just fall back silently to full replication if partial is not available. Currently partial replication is a bit of a hack, it will be turned into a proper solution over the next couple of months. So I'd rather leave it as is now.

PS: Would merge a PR with just the remove SSB changes.

PS2: I have now invited you to this repo so you can create branches here as well.

[KyleMaas]

This shouldn't be falling back to full replication. At least not if I understand the EBT API correctly.

What I did here was modify the initial sync step so that if partial replication of your ID fails, it falls back to requesting your ID to be replicated via EBT. Then once it has downloaded your own messages, it can build a graph and proceed as normal.

PS2: Accepted invitation. Thanks!

[arj03]

You might be able to get hops 1 in full, but then not the rest.

[KyleMaas]

I guess I'm not quite understanding. Since partial replication should only ever fail during onboarding (because otherwise EBT would be used like normal), "full feeds" should only contain your feed ID at this point. So if partial replication fails, it only has one feed to ask EBT to replicate. So effectively you're only EBT'ing hops == 0 for this first step.

[arj03]

Partial replication is also used if you follow someone, then then will lead you to get more hops 2 (partial). Anyway, the PR says: "you can onboard against clients which do not support partial replication" and I don't think that is a good idea right now because that means that you will get hops 2 in full.

[KyleMaas]

I guess that makes some sense. But isn't that already an existing problem, then? If you've already onboarded against a client that does support partial replication and then connect to another client that doesn't and EBT kicks in, you'd have the same situation. I guess I was figuring based on this message:

%on1/ZlTDH2fhxq+MGl4/dPlhQMqI5MlehjLOwzzYF4c=.sha256

...that onboarding was the main problem.

[arj03]

The way it currently works is that once you have synced with a feed then EBT on that feed works afterwards. The are 2 phases in the current replication: partial sync and EBT. I know there are problems with this approach but as I said in that thread we are working on a proper solution and I'd rather spend time on that :-)

[KyleMaas]

Okay. I'd really like to see ssb-browser be less of an isolated island, though, where it requires running specially-configured rooms and pubs (WebSockets not withstanding). And even other clients talking through a specially-configured room or Bluetooth or DHT can't actually be talked with. So the faster we can get that fixed, the better. What can I do to help with that?

And once a replacement solution is developed, what is our plan for talking with clients that don't use that or haven't been upgraded? What happens if someone's running an old version of Manyverse that doesn't support partial replication (like current versions) and a future version of ssb-browser tries to sync with them over DHT?

Would it work to do EBT sync with anything that's within hops distance and then use partial replication for profiles past that horizon?

[arj03]

@KyleMaas while I want to stress that all of this are still proposals and are thus subject to change, the current proposal that I still feel is what we'll end up with is this. I'm currently rewriting the meta-feed proposal to make it more clear and with better upgrade path. I hope to have that done tonight. Would love your feedback on the proposals. We might also end up implementing a set replication algorithm.

The current plan is to have the proposal ready for April and then start implementing stuff then. It will probably follow the same development model we used for ssb-db2 where the current code is split out of browser and then refined in it's own module.

As you can see from the proposal it will probably fall back to full replication if it can't do partial replication, but I think that should be a choice of the user. Like I don't want that to happen to on my phone running ssb-browser.

[KyleMaas]

Cool. Gives me something to read. Thanks!

[arj03]

Once this is merged, we'll move over to that way of doing partial replication

[KyleMaas]

Awesome! Really looking forward to the progress on this.