Add multi-tenant support

I.e. each Electric server can be serving arbitrary number of Postgres databases (though we'll probably want to have a default limit).

Background

There are many use cases for multi-tenancy. A self-hosted Electric server might have a staging and prod database that it fronts. A company might have multiple production databases they sync from. Electric Cloud will need this to efficiently host large numbers of customers.

This will be a breaking change as we'll need to include the database_id in the shape url.

Proposal

Right now we start the server with a DATABASE_URL environment variable to set the backend.

To add multi tenancy, the server will start with no connected databases. Instead we'll create an admin API for adding and deleting backing instances.

Each instance needs a Postgres connection string and an id. If an ID is not included, a UUID will be generated for the id.

Managing databases

You can start the database with an ADMIN_TOKEN to authenticate admin API calls. If one is provided, API calls must include it in the Authorization header.

To add an instance:

curl -X POST http://localhost:3000/admin/database \
-H "Content-Type: application/json" \
-d '{
  "name": "foo",
  "DATABASE_URL": "postgresql://username:password@host:port/database"
}' \
-H "Accept: application/json"

Can we complete setting up the new backend db within the span of the request?

To delete an instance:

curl -X DELETE "http://localhost:3000/admin/database/foo"

Can we complete cleaning up the deleted backend db within the span of the request?

Shape API change

Right now the path is /v1/shape/${baseTable} with query parameters.

With multi-tenancy, it'll be /v1/${database_id}/shape/${baseTable}.

We'll eventually want per-tenant controls e.g. max shapes, max shape log size, etc. that'll probably come along with https://github.com/electric-sql/electric/issues/1529

Logs

We'll need to add the db_id to the start of logs. This would also be a good time to add a JSON log option for easier parsing.

electric-sql / electric