Allow experimental flags to be enabled per user

[erikjohnston]

It would be useful to allow experimental flags to be enabled per user, to allow those with matrix.org accounts to test features with their real account without having to enable it for all matrix.org users.

Not all experimental features can be enabled per-user, e.g. experimental login flows wouldn't make sense to enable per-user (since the requester hasn't logged in yet!).

My proposal would be to store these in the database and add admin APIs. The rationale being that then we can (potentially) allow users to self-serve opting into experimental features.

[H-Shay]

A question comes to mind when looking at this: we currently enable/disable experimental features in the experimental section of the config. Would adding a table (I am imagining a new table called experimental_features or something with columns for each feature and a 1 or 0 for enabled/disabled per user added to the table - presumably users would add themselves to the table through the API?) replace the experimental config for the features moved to the table, or would they interact and if so, which one would take precedence?

[H-Shay]

The admin api requires that one have a server administrator access token and we suggest that the admin api be hidden behind a reverse proxy (https://matrix-org.github.io/synapse/latest/usage/administration/admin_api/index.html#making-an-admin-api-request), in this case would we make a public API for this instead and ask the user to provide something like their account access token?

[clokep]

I am imagining a new table called experimental_features or something with columns for each feature and a 1 or 0 for enabled/disabled per user added to the table

This was essentially the design I was thinking of, yes. (Although please use booleans and not 1/0.) The other potential design to consider is a table which has (user_id, feature, enabled) tuples. The benefit to this is that you don't need to keep adding/removing columns when features are added or removed. There are likely other pros/cons to consider between the two also.

presumably users would add themselves to the table through the API?

The admin api requires that one have a server administrator access token and we suggest that the admin api be hidden behind a reverse proxy (matrix-org.github.io/synapse/latest/usage/administration/admin_api/index.html#making-an-admin-api-request), in this case would we make a public API for this instead and ask the user to provide something like their account access token?

I had envisioned that it would only be modifiable by admins, we likely want to add an admin API for it to avoid people touching the database directly. I think this removes the need to worry about how "normal" users would access it.

This doesn't match what @erikjohnston said above about it being self-serve. I think I'd start by only allowing admins to use it, we could open it up (or add a new API) in the future if that's needed.

There's the /_synapse/client path which is generally routed to a public URL, so that could be an option for a non-admin path.

A question comes to mind when looking at this: we currently enable/disable experimental features in the experimental section of the config. Would adding a table replace the experimental config for the features moved to the table, or would they interact and if so, which one would take precedence?

I had assumed it would completely replace the config section, but maybe there are benefits to keeping it around? If we kept it around I think it would be something like:

flowchart TD
    Start((Start))
    Config{Is enabled\nin config?}
    Table{Is enabled for\nuser in table?}
    Enabled
    Disabled
    End((End))

    Start --> Config
    Config -- No --> Disabled
    Config -- Yes --> Table
    Table -- No --> Disabled
    Table -- Yes --> Enabled
    Disabled --> End
    Enabled --> End

[H-Shay]

Keeping the config adds complexity for an unclear benefit, so I'd also be inclined to replace it as well. Thanks for answering my questions, your perspective is very helpful!