turt2live / matrix-dimension

An open source integration manager for matrix clients, like Element.
https://dimension.t2bot.io
GNU General Public License v3.0
433 stars 110 forks source link

Add BigBlueButton widget support #367

Closed anoadragon453 closed 3 years ago

anoadragon453 commented 4 years ago

This PR adds BigBlueButton, an open-source web conferencing system, support to Dimension in a similar style to Jitsi.

However this implementation differs slightly to Jitsi in that there isn't a BigBlueButton library that we can just give a div and URL to and get a meeting out of. Additionally, there's an extra layer of complication as presenters, when creating a conference, interact with a separate service called Greenlight, which itself is a BigBlueButton client. The idea is one can use Greenlight's pretty UI to be able to create BBB conferences, access recordings, etc. It does this by using BigBlueButton's public API in the background.

Ideally we could also use this API, but alas when someone creates a BBB conference using Greenlight, they then share a Greenlight URL, instead of a BigBlueButton one. This is what Greenlight gives you, and what we expect people to paste in to their integration managers when setting up a widget. There's no simple way to translate a Greenlight meeting URL into a BigBlueButton one. Instead, the user is expected to load the Greenlight URL in their browser, Greenlight then asks you for a name to identify with, and it then interacts with BBB's API in the background and grabs a BBB meeting URL which you're redirected to, subsequently joining the meeting.

Now we could just pop the Greenlight URL into a widget iframe, the user would enter in a name, and call it a day. However, this UI isn't very easy to use when it's squashed inside a tiny Element widget UI frame, with cookie warnings all over the place, plus like with Jitsi we'd really like to be able to sync people's Matrix IDs, display names and avatars to the meeting, so the presenter can identify who everyone is.

The way this is accomplished is noted in a sequence diagram included in DimensionBigBlueButtonService.ts. It explains how the whole flow works from a user clicking "join meeting" to the meeting loading in the widget iframe, but the gist is Dimension has sprouted a new API endpoint which will transform a Greenlight URL into a BBB one. It does this by pretending to be a browser, loading up the URL on the server, supplying the user's Matrix ID and displayname to Greenlight (unfortunately avatars are not supported here yet, but adding support in Dimension once BBB's API gets it is trivial), which then returns us the BBB meeting URL. The widget gets that in the response, loads it up, and the user is in the meeting.

There's a couple other nuances which are accounted for here. One is that the user may try to join the meeting before it's actually started. This is in contrast to Jitsi where a meeting is ongoing as long as anyone is in it. Thus, we needed a way for the server to detect that a meeting hasn't started yet, and to communicate to the client that they need to wait. The server currently does this by parsing HTML (yay, no API!) and then returns a custom error code to the client. The client then attempts again to join the meeting after a small delay and eventually, once the meeting starts, the meeting URL should be returned instead.

The other little bit was that when a meeting ended, the user was kicked back out to the Greenlight UI, which looked odd as the user hadn't seen it up til this point. Ideally they would just see the "Join Conference" UI again ala Jitsi. To do this, we check when the iframe has finished loading a page, and then check whether this is a result of the meeting being loaded, or the meeting ending. Unfortunately, we can't just inspect the URL that the iframe has loaded due to browser security, so instead we use a heuristic that the first iframe URL change must be joining the meeting, and the subsequent one must be leaving the meeting. This seems to have worked so far in my testing :)

A final known issue is that Element Web special-cases jitsi widgets as being able to remain on screen even when the user clicks on another room. This will require some changes in Element Web to solve, but there shouldn't be any code changes require in Dimension once Element is fixed.

This is quite a chunky PR, so it's recommended to review commit-by-commit.

Addendum: It's worth noting that in order for BigBlueButton servers to allow their meeting content to be embeddable, they will need to make a slight change detailed here. Additionally, it's recommended for BigBlueButton servers to disable chat in their config files so that users will gravitate towards using Matrix for text communication instead.

fooness commented 4 years ago

Thank you very much for this pull-request as well as the detailed explanation.

A final known issue is that Element Web special-cases jitsi widgets as being able to remain on screen even when the user clicks on another room. This will require some changes in Element Web to solve, but there shouldn't be any code changes require in Dimension once Element is fixed.

Would be great, if that would happen soon™ …

I’ve got one question left, though. Is it somehow possible to reproduce the same with custom widget, so without using either Dimension or Scalar?

anoadragon453 commented 4 years ago

I’ve got one question left, though. Is it somehow possible to reproduce the same with custom widget, so without using either Dimension or Scalar?

Sure, although as described in the PR description there's kind of a sucky hack required to transform greenlight URLs to BBB join URLs, which may be a little hard to do strictly in the browser due to cross-site iframe issues (that's why we do it on the server here instead of in the widget code itself).

https://github.com/turt2live/matrix-dimension/pull/367/files#diff-b297645095d4937a1ec38cbb6d9a34b2R36-R38 (you will need to click Load Diff to see what is linked...).

fooness commented 4 years ago

That part in BBB is still somewhat (very) confusing to me.

In BBB, you can share the URL with others (e.g. https://bbb.domain.com/html5client/join?sessionToken=1aaj7i6yhynbd2sl … which you probably should not do.

Instead, you should share the URL that’s shown somewhere, though somewhat hidden-ish (e.g. https://bbb.domain.com/b/foo-xyz-123 … so we probably would need to be doing e.g. /addwidget https://bbb.domain.com/b/foo-xyz-123… and from there on just follow the frontend, I guess.

Just realized this is a pull-request and not an issue, sorry for starting this conversation here in the probably wrong place …

anoadragon453 commented 4 years ago

https://bbb.domain.com/b/foo-xyz-123 is a URL from greenlight, whereas the ...html5client... link is a BBB one. The hacky stuff we do here is to translate from the former to the latter.

Just realized this is a pull-request and not an issue, sorry for starting this conversation here in the probably wrong place …

This conversation probably belongs in an issue on the Element-web repo, as that would be where a non-integration manager based approach would need to be made. Feel free to create one for a local, jitsi-style iframe BBB widget :)

turt2live commented 3 years ago

I'm inclined to just accept this as-is given it has worked well enough for the people who have used it.