Invite code format: combining workspace address, key, and pubs

cinnamon-bun commented 3 years ago

What's the problem you want solved?

To invite someone to a workspace, you have to tell them:

A workspace address
One or more pubs
Maybe a workspace private key, if it's an invite-only workspace

That's too many things to copy-paste.

Is there a solution you'd like to recommend?

How can we combine all those things into a single string?

+gardening.xxxx?privatekey=yyyyyy&pubs=https://mypub.com,https://pub2.com

+gardening.xxxx.yyyyy|https://mypub.com|https://pub2.com

{"workspace":"+gardening.xxxx","privatekey":"yyyyyy","pubs":["https://mypub.com","https://pub2.com"]}

earthstar://gardening.xxx?...

???

cinnamon-bun commented 3 years ago

This issue was also discussed a bit in this thread

RangerMauve commented 3 years ago

Regarding the querystring approach, it'd be good to use repeating params like &pub=pub1.com&pub=pub2.com which could be parsed out with the url.searchParams.getAll('pub') API

RangerMauve commented 3 years ago

So something like earthstar://gardening.xxx?pub=pub1.com&pub=pub2.com 😮

Question about private keys, how do you feel about them being in the password section of URLs?

cinnamon-bun commented 3 years ago

This all feels so messy! Let's brainstorm more ideas.

@RangerMauve like this? Can the username be empty?

earthstar://:privatekey@gardening.xxx?pub=pub1.com&pub=pub2.com

I'd have two UX worries:

@gardening Looks like an Earthstar author instead of a workspace.
I'm also worried about the workspace privatekey being too easy to share by accident. If this is shown in a browser URL bar and someone copy-pastes it to share with friends, will they know they're giving out write access? Maybe there needs to be a special Share button that helps the user choose what permissions to share (like in Google Docs), and constructs an invite code for you.

These are only UX concerns so if the format is only used for a fetch() API and not shown in an URL bar it might be ok.

Also hmm, the author's name and secret is another contender to go in the username:password section.

Do we need query parameters for regular webpage use?

If we ever serve HTML pages directly from Earthstar (Beaker Browser style), will we need query params for their regular purpose related to the webpage instead of using them to provide pubs for the workspace?

I guess the # anchor part of a URL is more for the local page, and ? query params are for the "server", whatever that means in this case...

earthstar://gardening.xxx?pub=pub1.com#params-for-single-page-app-go-here

Invite codes also don't have to be URL-shaped if they're just for copy-pasting to friends, but I know for Agregore and the fetch API they do need to be URL-like.

RangerMauve commented 3 years ago

Yeah, I'm not sure how much you want to adhere to URLs vs other format. 🤷

I'm very biased when thinking about URLs because I really like how they're like a common denominator between projects and leave less stuff to be invented / not look like other things. At the same time, I see the value in creating your own identifiers that better suit the specific use case.

I agree that having the private key in the workspace could be an issue. Maybe private keys should be removed from URLs in URLbars by the browser after they've been authenticated somehow? Like, the protocol handler would prompt a person for the private key if they don't have it already and then save it internally.

Also, maybe invite links are treated differently from other ones? Like, the protocol handler in a browser or whatever would see that it's an invite, interpret the link using something that doesn't necessarily correspond to a regular URL and then redirect to the actual workspace. 🤔

Alternately, what about having everything be in the query string and treat it as more of a magnet link rather than a resource identifier?

Like earthstar:///?workspace=gardening.xxx&pub=pub1.com&privatekey=yyy?

sgwilym commented 3 years ago

Maybe it's worth looking at what invitations and onboardings to digital spaces currently look like, discussing what we like and dislike about them, and using any ideas from that to inform how to implement invitations?

Traditional centralised services:

Usually delivered by an email with a unique URL in it
- Thanks to it being a URL, all the user has to do is click it and the browser takes them to a page with a signup interface.
- The invitation and URL expires after the user joins

SSB:

I believe you can create a profile and start posting without any networking at all from within a client, it just won't be seen by anyone, so...
You need to be followed by someone to replicate you for SSB to be interesting
- Use an invitation code generated by a key (also only good for one use, I think?)
- This code needs to be copy and pasted into a client
- The invitation is used up after this
Sync with someone locally
- This is nice because you should just need to have your app open and on the same local network

Cabal:

You obtain a cabal:// URL, copy and paste it in, and enter a desired nickname
- Not used up, because the URL is the basically the secret to read and write messages to the Cabal, so you need to be careful how that URL spreads

Out of these three, only the traditional web service handles the case of a user starting from zero, and embeds a way to create an identity (i.e. the signup form). What could that look like with earthstar? Do you need to have bootstrapped yourself with a client and identity before you can do anything with the invitation your friend sent you?

SSB and Cabal let you start without needing any connectivity. That's nice. An Earthstar client could also let you create an author keypair and 'join' a workspace without any connectivity.

Web services and SSB have one time invitations, which means invitation codes can be handed out without worrying about them falling into the wrong hands.

I sort of got stuck here and didn't know what to do with this information. I have some half-baked ideas of pubs being able to serve web UIs to help completely new users generate an author keypair which they could export to use in a different client (and maybe do more invitation related things). But I noticed so many holes in my ideas as I was writing them that I thought it'd be better to stop here. Sorry if this is too much of a tangent. 😅

RangerMauve commented 3 years ago

What I'm kinda feeling from all this is that the invite isn't so much something you "load" but a thing you "use".

So I'd imagine an application existing that can use one or more workspaces for it's data. You load that application somehow, either trough a p2p web browser or something else, then you can paste in an invite which will almost "unlock" a workspace for you to use.

From the fetch side of things, I'd imagine it as USE earthstar://{workspace here} body: {invitation here}. Essentially, you use some sort of method name (I chose USE but I have no good reason for it), extract the workspace from the invitation, then use the invitation on the workspace to be able to access that workspace.

With that in mind the invitation can be whatever format. I'm still into URLs or a magnet-like format since it makes it easy to reuse URL for parsing / serializing, but I like all the other formats too.

cinnamon-bun commented 3 years ago

Many things to consider!

I don't think we can have single-use invites, otherwise some node (like a pub) will be responsible for keeping track of them. I've assumed that invites are more like Cabal links that contain the secret info you need.

Another totally different way to handle this would be:

A workspace contains an explicit list of authors who are allowed to write to it
To join
- Make up a keypair for yourself
- Send it to someone already in the workspace
- They add it to the list

I'm not a big fan of that one, it's a lot of back and forth and waiting

sgwilym commented 3 years ago

@cblgh just added an interesting fetaure to cabal-client: https://github.com/cabal-club/cabal-client/pull/63 and it made me wonder if there's a way to do expiring invitations with ephemeral docs:

🥐 wants to invite 🍄 to their private workspace.

🥐 creates an ephemeral doc on a public workspace, containing the invite formatted string that details the workspace and the private key.
🥐 passes the earthstar URL for the ephemeral doc to 🍄
🍄 enters this URL into their client which knows to fetch the content at that workspace + path and use the info inside to add a new workspace.
The client also deletes the document after usage.
And if it doesn't, the ephemeral doc disappears in five minutes.

On big problem: the ephemeral doc will still be viewable for those five minutes to anyone looking at the docs on the public workspace. ☹️

cinnamon-bun commented 3 years ago

@sgwilym interesting idea! I think it would be safer to use encryption (this is basically implementing private messages like Scuttlebutt):

Assume 🥐 and 🍄 are in a public workspace together already.

🥐 puts together the invite information for the other, more secret workspace
🥐 encrypts it to 🍄 and writes it to a doc in the public workspace in some reasonable location
🍄 decrypts it
🍄 uses the invite info to join the secret workspace

I've made an issue for this: #49 Add simple helper functions for encrypting & decrypting content

RangerMauve commented 3 years ago

Encrypting and decrypting stuff seems like a good tool to have for folks!

sgwilym commented 3 years ago

Sorry for derailing this issue with invitation chat! I'd like to come back to this...

react-earthstar has a component called AddWorkspaceForm. Right now it's two fields: one for the workspace name, one for the 'initial' pub. It would be great to have a single field that takes a URL which could encode a workspace and multiple pubs in one go (it would be even nicer to be able to add multiple workspaces and their pubs all at once, but I think that's a different issue...)

@RangerMauve's last comment suggested a format like this:

earthstar://?workspace=gardening.xxx&pub=pub1.com&privatekey=yyy

How do you feel about this format now @cinnamon-bun?

cinnamon-bun commented 3 years ago

@sgwilym @RangerMauve

Ok, let's do the parameter style of URL for invite codes. Does this look ok to everyone? We can iterate on the details later.

Earthstar Invite Code Specification v1

earthstar:///?workspace=+gardening.abc&pub=http://pub1.org&pub=https://pub2.org&v=1

Three slashes up front, because the traditional URL format is scheme://location/path?query and when you remove the location (which we're not using) you get three slashes.

Query parameters

Use the traditional rules for encoding and parsing query parameters.

workspace=+gardening.abc -- May occur zero or one times. A workspace address including its + character.
pub=https://pub1.org -- May occur zero, one, or many times. Include the scheme of the pub URL. For now pub URLs are not allowed to contain their own query parameters until we specify how to safely encode them.
v=1 -- Invite code specification version, always an integer

The order of parameters is not significant. Recommend putting the workspace first as it's the most salient to the user, and v last.

Intended use

This is not a URL meant for continuous display in a browser; it's an invite code meant to be given to someone, "used", and discarded. The UI should not keep it around after it's used.

Unfortunately invite codes can't be limited by time or number of uses; they remain valid forever.

User interfaces can have a "share" tool that generates invite codes.

Users SHOULD be able to decide which pubs to include in the code, since some pubs may be secret, only available on localhost, etc.
Once we eventually have workspace secrets, users SHOULD be able to include or omit the workspace secret. An invite code without a secret will allow people to host the data but not read or write to the workspace.
The UI MUST explain the privacy consequences of the invite code.

When using an invite code,

The UI SHOULD show a prompt with an easy-to-understand summary of what is about to happen, including a list of pubs.
Users SHOULD be able to toggle the new pubs so they can avoid adding some of them.
The goal is to avoid making network requests to new pubs until users have approved them.

Future plans

workspaceSecret=bj0q9j0039j -- Zero or one workspace private keys (for invite-only workspaces), encoded in Earthstar-flavored base32. Only useful if workspace is also specified. See #7 for details.
room=... -- Zero, one, or many addresses of room servers, if we ever have those.
swarm=... -- Zero, one, or many Hyperswarm discovery keys for finding p2p peers. Normally we'll use a hash of the workspace as the primary discovery key, but you could also make additional private swarms this way
vouch=... -- Maybe a signature from the person creating the invite, to help track who invited who? Need to think about this one.

sgwilym commented 3 years ago

Aye! I'm quite excited to start making these components in react-earthstar, great calls on UI @cinnamon-bun

sgwilym commented 3 years ago

Question: what are the use-cases for an invitation code with pubs but no workspace specified?

cinnamon-bun commented 3 years ago

@sgwilym

Question: what are the use-cases for an invitation code with pubs but no workspace specified?

It's a way to specially mark up a pub URL so that we can recognize it as a pub URL.

I don't know when exactly we'd use this. I thought it might be useful for browsers like Agregore where we can register our own earthstar:// handlers.

cinnamon-bun commented 3 years ago

I've made a wiki page summarizing and formalizing this Invite Code format.

https://github.com/earthstar-project/earthstar/wiki/Invite-code-format

Powersource commented 3 years ago

You sure you need the 3 slashes and 1 wouldn't be enough? My (limited) impression is that the // is only needed when pointing at a location. See e.g. email:person@example.com not having any slashes.

cinnamon-bun commented 3 years ago

@Powersource I'm not completely sure!

This may be the difference between a general URI vs. specifically a URL, if that distinction is even a thing anymore.

RangerMauve commented 3 years ago

Personally, I think that a workspace is a location and should be put in the hostname, same as in hyperdrives. This plays well with how browsers do origin management in that an Earthstar workspace could be an origin which is separated from other workspaces.

cinnamon-bun commented 3 years ago

For reference from the wiki page, here's the invite format we're using now.

earthstar:///
    ? workspace=+gardening.abc
    & pub=http://pub1.org
    & pub=https://pub2.org
    & v=1

It was meant to play nicely with other Earthstar URLs we invented in the future, and also it was not intended to be used as a persistent URL in a GUI but rather as a brief carrier of information for copy-pasting, more like an Intent or something.

The more general purpose URLs in earthstar-fetch look like this:

earthstar://+gardening.xxxxxxxxx/path/to/key.json

earthstar://WORKSPACE/PATH?QUERY

I agree with @RangerMauve that the workspace makes sense as a location. It might be time for us to change the invite code to match this usage. But I'm not sure where to pack in the pub URLs.

Idea for invite format v2?

earthstar://+gardening.xxxxx/?invite&pub=https://pub.com&pub=https://otherpub.com&v=2

sgwilym commented 1 year ago

This format needs to updated for share secrets.

I like moving the share address to the host name, it means this URL format is easier to re-use for different purposes (e.g. invites and resource locators), and also because + in the search params was always interpreted as a space. Not so in the hostname!

So an example of a v2 invitation code:

earthstar://+gardening.xxx/?invite&server=https://server.com&server=https://otherserver.com&secret=xxx&v=2

There must be an invite parameter (with no value)
There may be one or more server parameters, where the value is the URL of an Earthstar server sync endpoint.
There may be a single secret paramater, where the value is the base32 encoded secret of the share keypair.
There must be a v parameter with its value set to 2.

Earthstar core should have utilities to encode and decode these invitation URLs. SharedSettings should also be able to ingest and create codes too.

RangerMauve commented 1 year ago

Nice, that's great to hear. :)

Might want to URL encode the parameters to escape special characters. I think the URL class in JavaScript can do that for you automagically.

sgwilym commented 1 year ago

I think we should be okay. The share address and secret are always encoded as base32, and the server URLs should be safe as they're URLs.

Or is there something I'm missing @RangerMauve?

Powersource commented 1 year ago

moving the share address to the host name

Dat did this too, but I have some memory that that ended up with compatibility problems because hostnames have length restrictions in some cases or something.

sgwilym commented 1 year ago

@Powersource oh, word? Any idea what those cases were?

MDN says a TLD can be 63 characters long, and our pubkeys are always 53 chars long. The name (e.g. gardening) can be 15 characters max. So altogether it should fit in under the hostname limit of 253 chars handily.

Did dat have super long hostnames (not TLDs) too?

Powersource commented 1 year ago

If you have those length restrictions then things might be fine. A quick search makes the dat hostname look at least 64 chars long.

I think @christianbundy might've been the one who pointed out this issue.

RangerMauve commented 1 year ago

Or is there something I'm missing @RangerMauve?

Just mentioning it to keep with the URL standard and having less exceptions if/when earthstar starts getting used in other programming languages. Easier to have valid ish URLs where querystring params are URL encoded and parse with URL than implement URL parsers in new environments from scratch

RangerMauve commented 1 year ago

Regarding length restrictions, it's true that hostnames are constrained to 63 chars, but in practice it hasn't been an issue for me. Might just be troublesome if you register with IANA or whatever. Could also cheese it and say it's a URI instead of a URL to sidestep the issue

christianbundy commented 1 year ago

oh yeah! I forgot about this: https://github.com/dat-ecosystem-archive/dat/issues/1050

great memory @Powersource

earthstar-project / earthstar