Open togir2 opened 1 year ago
This idea has been in my head for a long time, great that you want to tackle it.
I would like to add Websoket support to the Django-Api via channels.
I agree, I wonder what backend we want to use though, redis seem to be the obvious answer, but maybe we can leverage the existing services (rabbitmq/postgresl) instead of adding a new one. Or maybe we should simply add redis as it might be useful in the future anyway.
I would like to have two sockets one "public" to which external clients can connect to get "now-playing" and "schedule-updates", and one "internal" for the webui to replace the polling.
Sounds like a good idea.
One problem we need to solve before doing this, is to properly handle authentication and authorization in django.
Right now, we only allow read only access on some resources for the public, read only with the api key for internal communication and the read/write for the admin, but maybe @paddatrapper can correct me on this one, as we might support a bit more.
In addition, we might have to authenticate twice (legacy + api) for any user using the UI, or setup a shared session store and only log in once. I would love to only log using the api and use a custom shared php session store for legacy https://github.com/libretime/libretime/issues/1788. Maybe we also use token based auth instead.
I agree, I wonder what backend we want to use though, redis seem to be the obvious answer, but maybe we can leverage the existing services (rabbitmq/postgresl) instead of adding a new one. Or maybe we should simply add redis as it might be useful in the future anyway.
I do not think we need redis, there is a backend for rabbitmq https://github.com/CJWorkbench/channels_rabbitmq
Right now, we only allow read only access on some resources for the public, read only with the api key for internal communication and the read/write for the admin, but maybe @paddatrapper can correct me on this one, as we might support a bit more.
The permission structure is based on the same groups as the UI - Guest, Host, Programme Manager and Admin. Specific permission handling isn't done for Admin, as they designated Django super admin access. The rest are defined in https://github.com/libretime/libretime/blob/main/api/libretime_api/permission_constants.py and https://github.com/libretime/libretime/blob/main/api/libretime_api/permissions.py
I did some thinking and have a proposal on how to intgrate and couple channels into the django-app
The rough idea idea is:
The channel will transport messages that are encoded as json. All messages have a top level "type" field to distinguish them.
After connection client can send two events:
{"type":"subscribe", "group":"groupname1" }
{"type":"sunubscribe", "group":"groupname1"}
For troubleshooting the client will receive a response to either of these events:
{"type": "subscription_status": "subscriptions":["groupname1", "groupname2"]}
When the server receives a "subscribe" message it will check if the user (if there is a user) has permission to view the resource. If the user has the permission the client will be added to the "group" of the requested model.
We can overwrite the save/delete methods of the Django models. The methods will then notify the "group" model.__name__ about the changed data . The message a client will receive could look like this:
// action could be add, update, delete
{"type": "data", "action": "delete",, "group": "group_name" "data": json(model_values)}
We can either do this individually in every model, provide a class decorator or use a wrapper class that overrides the methods. I prefer the wrapper class.
There could be custom groups that do not map to a django model. There should be one file which list all the custom groups. E.g. a "live_info" group that pushes combined data about the current show and track. They should start with a prefix (that cannot be a class-name in python e.g "event-*") to avoid having a name collision with a model class later. As custom group permissions can also not checked via the Django permission convention it must provide a permission.
The permission for a group is of form "view_modelname" for models or the specified one for the custom groups. We can check with user.has_permission() if a user can be added to a group.
There should be a page in the docs that list all available groups and the data-format. The data should be generated from the models classes. The custom endpoints would need to provide there own definition class.
I do not think they can be integrated into the openApi schema.yml https://github.com/OAI/OpenAPI-Specification/issues/55#issuecomment-1050102436
There is https://www.asyncapi.com/ which seems it can do a similar thing for web-sockets. We could then (in future) generating Datetypes for the client to ease on the development there.
Is your feature request related to a problem?
The webui is using polling to update the data its showing. E.g. the /Schedule/get-current-playlist/ endpoint is called ~5 sec. to check for changed playout information.
The Library and "Scheduled Shows" are also polling but on a much longer interval, which can lead to inconsistent views when multiple people (or tabs) are using Libretime simultaneously. E.g. one is adding content to a show via the "Calendar" and the other one (not noticing, in his view the show is still empty) via the "Dashboard". This will result in duplicate content.
Describe the solution you'd like
I would like to add Websoket support to the Django-Api via channels. I would like to have two sockets one "public" to which external clients can connect to get "now-playing" and "schedule-updates", and one "internal" for the webui to replace the polling.
I do not want to replace the existing REST-Api. I do think the Websockets only make sense for update notifications, not the initial loading/updating data. Also I think the REST-Api is easier to integrate to external tools.
I took a look through the webui and worked out the following events:
The events should contain the changed data for the webui to check if it should update.
The clients should be able to choose which events they want to receive.
Describe alternatives you've considered
An alternative to the Django-Channels would be to use RabbitMQ-websockets but then we would need to write cusom code to send the messages. Also I do not feel well exposing the RabbitMQ to the internet.
Also a Graphql endpoint could combine the "REST-Api" with an subscription feature for updates. But as there is already a REST-Api I don't think we should have an additional api with "similar" features. Also for the "subscription" the Django-Channels are needed to.
Additional context
The Websockets can only work for the items that are handled in the Django-Api so this should be added incrementally as the api gets migrated.
As a POC I would like to add the Websocket updates for the "live-info-changed" event as I think this is the easiest to implement in Django and webui.
I would appreciate to have some feedback for this idea :)