Replay recordings

[gallypette]

Hi,

After recording a session with several participants and screen shares, galene produces a .webm file for each video stream.

Do you have any plans/ideas about how to mix these files together to replay a session? My use case is to replay a lecture so student who missed it can see it as it was.

[jech]

Yes, the functionality is needed. I don't think it should be part of the server, though, it should be a separate utility. I'm not quite sure what the right approach is.

• A piece of Javascript that replays the session, synchronising the different streams?
• A separate utility, that combines the different streams into a single video that can be uploaded to a Peertube instance?
• An XML file that can be imported into Kdenlive to make postprocessing automatic?

I'm open to any ideas that don't involve bloating the server with media processing.

[gallypette]

I am wondering whether the easiest would be to have a simple script that lists a recording's folder and use the files' timestamps to spawn ffmepg processes. Each ffmepg process would stream the webm's content via webrtc in a 'replay' room that would be not much different than regular rooms in the end.

I am pretty sure there are a lot of caveats to my idea though :)

[adulau]

I think a separated tool would be the best. ffmpeg concat should work. The only issue is to find the proper timestamp 0 as the screen sharing might start later.

[hferee]

My quick thoughts on your various ideas:

* A piece of Javascript that replays the session, synchronising the different streams?

Easyly done and has the advantage of being self-contained. One can have about the same user interface as a live galene meeting. No server-side processing, but relies on the server bandwith. In my opinion it is OK for a small seminar, but not for large conferences with many downloads.

* A separate utility, that combines the different streams into a single video that can be uploaded to a Peertube instance?

That would be quite nice, especially for recording big events! But where would the processing happen? On the server means bloating it ; offline would use an option below, and a manual import into a peertube instace would suffice.

* An XML file that can be imported into Kdenlive to make postprocessing automatic?

That can be done as well, using the MLT framework that is behind kdenlive and other video editing software. It requires to identify (server-side) the slides, and probably the speaker. Then there are a lot of design choices about the placement of each stream.

I am wondering whether the easiest would be to have a simple script that lists a recording's folder and use the files' timestamps to spawn ffmepg processes. Each ffmepg process would stream the webm's content via webrtc in a 'replay' room that would be not much different than regular rooms in the end.

This has the "bloating the server" drawback and requires a very powerful server to be able to handle the encoding in real time. It also wastes a lot of computing power by reencoding everything for each client.

I think a separated tool would be the best. ffmpeg concat should work. The only issue is to find the proper timestamp 0 as the screen sharing might start later.

Making timestamps relative to the first is pretty simple. The issue is that the final video should contain several video and audio streams at the same time, depending on the usage (slides, screen sharing, speaker, chairman or questions for a seminar for example).

[jech]

It seems to me that there are two distinct scenarios here:

• users who want to view a recorded session;
• group administrators who want to generate a useful video that they can upload to a web server or to some "tube" video distribution service.

We need to serve both.

[SteveClement]

As far as I understood you record every "participant" in their separate .webm

Would it be a good first step to investigate if adding timecode(s) to the created webms?

According to the container docs there are quite a few timecode options present.

A more hacky way would be to just bang a Unix timestamp somewhere in each .webm and do some magic pre-processing when combining all the files.

But maybe you already foresee this, I have difficulties reading diskwriter.go (well I cannot really read go is the issue)

I wanted to see what library actually produces the webm files.

[jech]

As far as I understood you record every "participant" in their separate .webm

Yes.

Would it be a good first step to investigate if adding timecode(s) to the created webms?

Yes.

A more hacky way would be to just bang a Unix timestamp somewhere

The timestamp is encoded in the filename, in ISO 8601 format. I agree it's a hack.

[SteveClement]

The timestamp is encoded in the filename, in ISO 8601 format. I agree it's a hack.

Good enough for me ;)

Thanks.

[deleolajide]

• users who want to view a recorded session;

I have implemented a client-side recorder and viewer for Jitsi Meet generating an HTML page with embedded <video> tags for each participant's media.

I generate a webVTT file for all chat messages and any custom captions/sub-titles that were added by the conference admin. I had to generate some JavaScript as well to sync all the videos to the controls of the first video. This allows the viewer to control the playback from a single video control panel.

This approach could be adapted here and certainly work better with the server-side generation of the webm files.

[theRAGEhero]

This feature would be awesome to add to Galene. Right now there isn't really any software that is easy to deploy and that record. Jitsi needs Jibre, if Galene could generate a usable recording a lot of people would use it. Right now recording conferences mostly works on commercial software... Thank you.