A web frontend for `ayb`

[marcua]

Some open questions

• How should authentication with the API happen? Do we need session-based auth, or is there a way to rely on the API keys/secrets?
• Can each ayb instance host its own web frontend, or is the web frontend project distinct from ayb?

[sofiaritz]

Can each ayb instance host its own web frontend, or is the web frontend project distinct from ayb?

This is how I envision the relationship between ayb and aybWeb (let's call the web frontend like that for clarity purposes) in the future:

1. ayb should only handle two things: the server and the CLI client.
2. aybWeb could be the "recommended web frontend", but it shouldn't come bundled with ayb.
3. ayb could offer "official" Docker Compose files with popular frontends:
   • A "bare" option should also be given, where only the server is installed.
   • These frontends should meet the following requirements:
     • It has graceful degradation (in an ideal world, an old ayb server should be able to use the latest frontend version).
     • It has an open-source license.
     • It is actively maintained and its maintainers can be trusted.
4. aybWeb could be a sub-project of ayb, but it should still effectively operate as a different project, just with better communication and coordination channels than a fully independent project.

This vision only applies once 1.0 is reached and once the project has gained some traction and more frontends are created.

My vision about the present is the following:

1. ayb should only handle two things: the server and the CLI client.
2. aybWeb is an independent project maintained by me (and you, if you're interested once I publish the source :p).
3. The ayb documentation/repo links to aybWeb, aybWeb will provide two options: self-hosted (an aybWeb interface only for your instance) and community-hosted (instances of aybWeb that aren't tied to any instance, example of something similar https://phanpy.social/)

In both cases there is a clear separation between ayb and aybWeb when deploying them. This isn't something new, most Akkoma instances host both PleromaFE and MastodonFE, which means that users get to choose which one to use to interact with their instance. Once I start hosting an ayb instance, my idea is to have aybsrv.sofiaritz.com point to the ayb instance and ayb.sofiaritz.com point to the aybWeb instance.

How should authentication with the API happen? Do we need session-based auth, or is there a way to rely on the API keys/secrets?

That's a tough one, in an ideal world there are no bad actors and no mistakes so we don't need any authentication at all. :p I'm not sure I can answer this right now, in a typical email-password scenario the answer is easy: login with your email and password, an access token and a refresh token are returned, the frontend periodically refreshes the token, etc.

Right now ayb is based on email-token authentication, which I don't think is necessarily bad, but it relies too much on email, and email is not a secure way of transmitting and saving sensitive things (also, #217).

In my opinion, username-password-2fa(totp) with (optional) email is best, this means the following:

1. Instance owners can (but don't have to) require users to input their email and verify that their email is valid.
2. Requiring 2FA adds more friction and a bit more complexity, but the gains on security outweighs the friction.
3. Instance owners who do not require users to input their email addresses can still help with account recovery thanks to having two factors (password and TOTP) instead of one (password).
4. Instance owners who do not want to deal with emails (either because they don't want to store personal info or because they don't want to deal with email altogether) have that option available to them.

In either case, I think that it's best to have some kind of password-based auth in order to create short-lived tokens that would work like session tokens. These short-lived tokens could be JWT tokens, which means that #214 wouldn't be needed anymore because the information would be stored in the token and actions like creating new databases would refresh them (after verifying the password/2FA).

---

Another thing related to the frontend, one change that should be done in the API is how errors are sent. Errors sent by an API should be easily parsed by a machine, and right now most of them are strings with a debug representation of the AybError struct embedded. A more structured error format in the API would mean that frontends could help the users navigate issues more easily :)

---

Yeah, 5000 characters weren't going to fit in a Mastodon DM haha.

First of all, thank you for your time, I'm really excited to work and help with ayb :) Second, some things here may not be really clear (the fact that I don't have much time and that English isn't my native language doesn't really help), feel free to ask me any questions, I'll update this comment accordingly if needed And third, I feel like the authentication part may sound a bit picky: that's not how I want the message to come across at all, I think that your approach to authentication is pretty good (let's not forget that this is a project that's in its very early development phases, the fact that here is any authentication at all is pretty outstanding haha) but I can't find a way to re-write that paragraph in a way that satisfies me, so I've just written another one to explain how I feel in that one :p

[marcua]

Thank you for your thoughtful response, @sofiaritz !

• Re: ayb and aybWeb: I'm largely on board with your 1.0 and present-term vision. I worry a little bit that the loose coupling (which I agree has many other benefits) will add complexity to a new user's mental model as they try to understand ayb, and add operational complexity in having to run multiple services to get up and running. That said, I suspect a lot of this concern can be addressed by making it possible to optionally package aybWeb frontends with an ayb server, and also know that we can always reconsider later and more tightly couple the systems if it benefits the user. The bigger threat to aybWeb and ayb right now is not finishing the v1 of each, so let's solve that problem first using the model you propose :).

• Re: AybError not being machine readable: I totally agree. This was bullet 3 of #103 , but now it's bullet 1. Thank you for the push!

• Re: authentication and tokens:

  • tl;dr: I think if we agree on the cookie in which to store the ayb secret in aybWeb, most of the work will be on ayb to make a safe cookie. Shall we call it ayb_sessionid? I bet we can prototype this rather quickly, but also think it's not a blocker given your current workaround in the settings pane of your prototype.
  • We should differentiate between the authentication method (e.g., email, password, 2FA, etc.) from the proof of authentication (e.g., the tokens ayb currently uses, a shorter-lived session stored on a cookied, etc.). ayb already intends to support multiple authentication methods --- it supports email as one concrete type right now, but we can support others in the future. That's less important to how aybWeb will communicate with ayb than how aybWeb will store the token. Because the initial use cases were around a client sending API calls with a bearer token, the confirm endpoint returns the token and then stores a hash to verify the token on future requests.
  • The question for us is how ayb should best respond to the confirm request with the proof of authentication in a more browser-friendly way. I suspect it's as easy as having the confirm endpoint, in addition to responding with the API token in the body, also storing the token (or an encrypted version of the token) in a cookie that a browser can send back in future sessions. If we combine the cookie with some sort of shorter-term lease/expiration on the token, we'd get the desired result. You mentioned JWT as one way to encode the token. As long as the JWT wrapped the API token, that would fit into our framework. We could alternatively fernet-encrypt the API token, which has the benefit of not needing to introduce a third form of encryption to ayb, and has the benefit of already supporting expiration. The implementation details won't matter much to aybWeb --- it should just happily store the cookie and send it back with requests, and ayb can do whatever it needs to confirm the identity of the requester.

[sofiaritz]

I worry a little bit that the loose coupling (...) will add complexity to a new user's mental model as they try to understand ayb

From the perspective of a new user, there are instances with an UI and instances that "only" work with the CLI. A new user doesn't need to understand the relationship between ayb, the CLI, and webAyb.

The bigger threat to aybWeb and ayb right now is not finishing the v1 of each (...)

hahaha, exactly :)

We should differentiate between the authentication method (...) from the proof of authentication

Yeah, I got things mixed up there :p

---

About storing and returning the secret I think there are two paths we can take:

1. The Cookie route
   • The servers sets the cookie directly (I like ayb_sessionid), that cookie should be both HttpOnly and with SameSite set to Strict and potentially encrypted by the server.
   • This means that #214 should still be implemented.
   • This also adds some coupling between frontends and servers because there are now two kinds of authentication proofs: Bearer tokens and cookies (only for web-based clients).
2. The Authorization route
   • The server returns a list of authentication proofs (example), then each client decides which one to use. Every client would use the Authorization header, maybe with a prefix to help the server identify the kind of token (example).
   • This means #214 doesn't necessarily need to be implemented because the JWT already contains the data.
     • JWT doesn't encrypt the payload, instead it adds a signature that the server can check. This means that the JWT both serves as an authentication proof and as a data container.
   • In contrast with the Cookie route, the server only needs to check the Authorization header, which also prevents edge cases related to receiving a request with both a cookie and a header, and things like that.

I prefer the second one (with or without JWT), but I want to hear your thoughts on those two options :)

[sofiaritz]

Unrelated to the current discussion, but related to the web frontend: what is your vision for this web frontend? I just implemented a simple frontend with the endpoints I had available and a few assumptions I made :)

A more GitHub-like experience would require different endpoints and UI/UX decisions than a PlanetScale-like experience :p I know that you are more inclined towards a GitHub-like experience.

[marcua]

Re: cookie vs. authorization: I agree that the authorization route looks cleaner, but I'm (probably being silly and) stuck on one conceptual thing: how would a browser store the token? The bearer auth token setups I've seen have all been for cases where a backend stores a token/secret and attaches it to HTTP requests. In the case of a browser that I log into, don't I need to store a cookie so that when I reload or close the browser, the session/cookie is maintained?

Re: the web frontend

• I have a lot of thoughts on the frontend, but my primary one is that the v1 you created is infinity times better than the non-existent one we have today! :) It also matches the current state of the backend and data models, which will expand with time.
• I'm an incrementalist, and think that the experience you have for an entity (show the databases, drill into a database, query the database, eventually share the database with others) is the right one, and we can expand from there.
• Not to keep any secrets, the reason I envision a GitHub-style experience is that it will ultimately allow discovery/exploration/collaboration, where you can find people who are interested in similar datasets as you (e.g., "We all live in city X, and are building a collection of datasets for that city"). The limiting factor here is the backend: in order to realize the longer-term vision on the web frontend, we'll need to first add access control, sharing, publicly read-only access, organizations, and the ability to list entities, to name a few. :)

[sofiaritz]

how would a browser store the token?

localStorage. That's the way it is implemented right now in my WIP frontend, and I have deployed various services whose APIs use the Authorization header and the frontend stores the token in localStorage :)

my primary one is that the v1 you created is infinity times better than the non-existent one we have today! :)

Hahaha, thank you :))

and think that the experience you have for an entity (...) is the right one, and we can expand from there.

That's great to hear!

Re: GitHub-like experience. Alright, I think I fully understood the idea now. I'll do some prototyping around some GitHub-esque UI tailored to the "database-style". I think that having things like the ability to run non-destructive queries on public databases could be nice... I'll do some testing on my fork and report back to you :)

[marcua]

localStorage

No problem if it works for the v1, but when I've seen localStorage used, it's been more for "runs entirely in the browser, copy/paste a token here"-style experiences. I've seen more sessions/cookies for browser-based auth, but I don't think any of this is a blocker to prototyping the concept regardless. For what it's worth, I suspect the existing API is fine if the intention is to store an API key in localStorage.

One detail this is making me realize is that we'd have to figure out (regardless of key management) is how to tell ayb to send a confirmation URL to aybWeb via email rather than a command line confirmation link. Would be easier with tight coupling, but nothing a little configuration/templating can't fix :).

I'll do some prototyping around some GitHub-esque UI

This is amazing, but don't feel pressure to get it all done in a v1. That said, the read-only queries thing is exciting, and I was hoping to make progress on public read-only access in the API after figuring out some details about isolation (#41).

[sofiaritz]

I was hoping to make progress on public read-only access in the API after figuring out some details about isolation

That's great! Keep me updated :)

but when I've seen localStorage used, it's been more for "runs entirely in the browser, copy/paste a token here"-style experiences

Not at all! localStorage is very useful for non web specific APIs. The following diagram describes a complete log-in and authenticated action sequence that works just like a cookie-based website from the perspective of the user.

I haven't implemented a login page based on email-password credentials in aybWeb because that endpoint doesn't exist yet, but the idea is to implement that before the initial public release.

sequenceDiagram
    User->>Frontend: Log-in info (username, e-mail, password, etc)
    Frontend->>API: Log-in info
    API->>API: Create a token
    API->>Frontend: Send the token back
    Frontend->>Frontend: Store the token in localStorage
    Frontend-->User: Redirect the user outside the login page
    User->>Frontend: Run authenticated action
    Frontend->>Frontend: Retrieve the token from localStorage
    Frontend->>API: Request to run authenticated action, sends the token
    API->>API: Check that the token is valid
    API->>API: Run action
    API->>Frontend: Return the result of the action
    Frontend-->User: Display the result

[marcua]

That's a very helpful diagram, thank you! Given you've got things working end-to-end, I propose we not spend time on another session/authentication mechanism just yet. If anyone wants to build a web frontend that doesn't live entirely in the browser, or if we ever want to support expiring keys, we can revisit it at that point. I feel like the remaining loose end is the first portion of your diagram:

sequenceDiagram
    User->>Frontend: Log-in info (username, e-mail, password, etc)
    Frontend->>API: Log-in info
    API->>API: Create a token

Right now, when the frontend calls the API to register, the user gets an email with a short-lived confirmation token. The email says to use the command line to confirm the registration. We'll want ayb to optionally offer another (templated) way to add the confirmation token to a URL (e.g., "To complete your registration, visit https://ayb.sofiaritz.com/confirm/{token}"). One idea is to support an optional confirmation template string that defaults to the command line confirmation instruction but can be configured in ayb.toml to send the user elsewhere. When the frontend receives a request at that URL, it sends it to the API, and gets access to the token. Voila!

[sofiaritz]

One idea is to support an optional confirmation template string that defaults to the command line confirmation instruction but can be configured in ayb.toml to send the user elsewhere

I like the idea :)

Something like this:

[email.templates.login]
confirmation_url = "https://ayb.sofiaritz.com/confirm/{token}"

That would allow us to do things like this in the future:

[email.templates]
html = true
footer = "<a href=\"https://github.com/marcua/ayb\">ayb</a> · <a href=\"mailto:sofi@sofiaritz.com\">Support</a>"

[email.templates.login]
confirmation_url = "https://ayb.sofiaritz.com/confirm/{token}"

[email.templates.reset_password]
reset_url = "https://ayb.sofiaritz.com/reset/{token}"

[marcua]

Now that you have it all working, I'm going to resolve this issue, but feel free to re-open if we need to think through the design more. Thank you for making this happen! :)