[Feature]: Zoom integration

[mtdvlpr]

Is your feature request related to a problem?

Altough M³ makes life so much easier for the audio/video brothers, there's still a lot of manual tasks that need to be memorized, especially the Zoom actions before and after meetings (join/leave computer audio, mute all, (dis)allow to unmute). With a Zoom integration these steps could be automatized which would take M³ to the next level!

Describe the solution you'd like

In an ideal setup, I would want a Zoom integration mode (similar to the OBS integration mode), which would require a name, meeting id and password. When enabled, I would like to at least see the following options with automated actions:

• Start meeting: disallow unmute themselves -> mute all -> join computer audio -> turn on mic and camera -> optionally spotlight camera (could be automated based on meeting time, similar to auto stop shuffle music)
• End meeting: remove spotlight -> leave computer audio -> turn camera off -> allow to unmute themselves (could be automatic when you leave presentation mode)

Some extra options:

• Give mic: participant select -> unmute participant -> lower hand
• Take mic: mute participant (prevents misclicking and unmuting someone else instead)
• Zoom participant mode could be enhanced with a participant select which would pin the selected participant for you

Describe alternatives you've considered

jwzoom.de has Zoom integration, but it's in German.

Additional context

There are multiple tools Zoom provides, but it's uncertain which tool we'd need and how we'd have to implement it. There's Zoom Meeting SDK, Zoom APIs, Zoom webhooks and a beta Zoom WebSockets.

[joaochris]

@mtdvlpr It would certainly be great! Know this one? I've been using it since the Zoom meetings started, it's a masterpiece :D https://github.com/edunsouza/zoom-custom-features

[mtdvlpr]

@joaochris, how would you use it? I see a bunch of scripts, but no clear way of executing them.

[joaochris]

@joaochris, how would you use it? I see a bunch of scripts, but no clear way of executing them.

I copy the code from https://zoom.vercel.app/ and paste it into the console in a Zoom Web meeting.

[wifilogo]

@joaochris, how would you use it? I see a bunch of scripts, but no clear way of executing them.

I copy the code from https://zoom.vercel.app/ and paste it into the console in a Zoom Web meeting.

Wow! Powerfull!

[mtdvlpr]

@danielengels, anything you could help us with is great. Writing code, explaining the steps/tools necessary, anything else.

[danielengels]

@mtdvlpr Except the generator for the signature, everything is in the Javascript File. I use the webSDK from Zoom: https://marketplace.zoom.us/docs/sdk/native-sdks/web/component-view/reference/ As there are not every feature I need, I listen on the websocket and intercept the messages from the zoom websocket and send some messages to the socket and don't use everything from the API. There is also a Electron SDK: https://marketplace.zoom.us/docs/sdk/native-sdks/electron/sdk-reference/ Maybe we can also try, to use my site and you can load it in an own render process... but maybe also too hard to do. I will try also the next days to download the git and I will try it on my own. I used Vue and Electron some time ago, but just for some tests.

[mtdvlpr]

I think the easiest way would be to follow the instructions for a Vue app, like https://github.com/zoom/meetingsdk-vuejs-sample

[danielengels]

Sorry, I can't write the last days. I think a little about, how it would help to do in m³. I have two rough ideas:

1. you integrate the schedule of the meeting and connect media with the talks. Something like the plan on our workbook. First you had to chosse your own congreation zoom meeting user. And you can prepare the meeting and add the user(s) to the specific talk. If the talk starts, the "scene" toggle the audio of the zoom user(s) and put them into spotlight. If you play some media the spotlight switch back to the congregation zoom user and back after stoping the media. Then you can stop the "talk" and switch back. But that means, that for every meeting need to save the additional meeting outline. This could even maybe get saved in the media dir as a .meeting.json or something like this and could be even synced with the cong sever.
2. similar idea but withouth the meeting schedule. You can create scenes that are connected with the zoom users and you have one default zoom user for the congregation and then you can switch the scenes when they start. And they do the same as described before. It could be similar to the obs scene selector where you overwrite the default scene for a certain time.

[mtdvlpr]

@danielengels, no worries and thanks for the brainstorming! I like the idea of creating "scenes" on the spot. One, because I don't know how feasible it is to extract the meeting schedule from the jwpub file. Two, because you often don't know in advance which parts are going to be held through Zoom.

I think for now however, the focus should be on automating the Zoom steps before and after the meeting. When training brothers to help with the audio/video, I often hear that those steps are the most difficult. If we can automate that either x minutes before meeting or with a press of the button, it would be a great step!

[sircharlo]

@danielengels, no worries and thanks for the brainstorming! I like the idea of creating "scenes" on the spot. One, because I don't know how feasible it is to extract the meeting schedule from the jwpub file. Two, because you often don't know in advance which parts are going to be held through Zoom.

I think for now however, the focus should be on automating the Zoom steps before and after the meeting. When training brother to help with the audio/video, I often hear that those steps are the most difficult. If we can automate that either x minutes before meeting or with a press of the button, it would be a great step!

Agreed

[mtdvlpr]

We've got a working proof of concept! The next step is to figure out where to place the different actions. The first actions we want to support are automating the start and stop meeting flows. All ideas are welcome!

[danielengels]

Sorry, i was long time not here. How can i help?

[mtdvlpr]

Hi @danielengels, please take a look at #1303. There's a test version you can use. Any feedback is welcome and appreciated!

[danielengels]

Hi @mtdvlpr, I downloaded it and tried to use, but it was not possible. Maybe i do something wrong. I get the errorNoSocket Message. Do I have to prepare something?

[mtdvlpr]

@danielengels, in settings, you need to enable Zoom integration and at least fill in the name, id and password fields. When opening the presentation mode you should see a blue Zoom bar and in the Zoom meeting you should see that a participant with the configured name wants to join. That participant should be granted co-host privileges.

[danielengels]

yes, but after debbugging a little, i am sure it is the ZOOM_SDK_KEY and ZOOM_SIGNATURE_ENDPOINT because I don't have an .env. I saw the changes, because it can't connect to a websocket.

[danielengels]

yeah, it is the endpoint - sure, you changed it and i thought I could just use mine ;) but understand, I just changed the axios call a little

[danielengels]

Those who can read clearly have an advantage :D I just saw the branch and check the branch out and the .env was not set... but i had the jwzoom endpoint and it was not very hard to change it ;)

[mtdvlpr]

@danielengels, so you got it working without the errorNoSocket?

[danielengels]

yeah, the dev version (with a little modification to use the jwzoom-endpoint) and now also the much easier way by installing. But I can't really spotlight someone. They only get unmuted.

[mtdvlpr]

Do they have their video on?

[danielengels]

yes - i also thought first it had to do something with this. But it is not. Both version my local and the installed one does not work. the reason seams to be a second websocket from zoom which is not the control websocket... I guess you are doing the same like me and just using the last websocket established... but I don't have this second websocket to zoom. Maybe a newer version of the zoom sdk?

[mtdvlpr]

Okay. I did notice a second websocket, when trying to figure out why the security > unmute themselves wouldn't correctly get toggled. So you're saying both live simultaneously and only one of them is the correct one?

[danielengels]

yes, that seems to be the problem... on the websocket "hijack" i add that the url should includes "dn2" and then it works.

WebSocket.prototype.send = function (...args) {
        console.debug('send:', args)
        if (
          this.url.includes('zoom') &&
          this.url.includes('dn2') &&
          !window.sockets.includes(this)
        ) {
          console.log('socket', this)
          console.log('sockets', window.sockets)
          window.sockets.push(this)
        }
        return originalSend.call(this, ...args)
      }

[danielengels]

Another question: When a picture/video is played by mmm nothing changed on the spotlights. Shouldn't it change to the host when you start sharing and when you stop change back to the participants spotlight? Another question - what do the "automate audio" in the settings? Btw. the "subheaders" feature is awesome!

[mtdvlpr]

@danielengels, you're right! I hadn't considered that since we made the OBS Zoom scene for that purpose (Zoom parts with media). I suppose when we implement that, the OBS Zoom scene becomes obsolete.

Another question - what do the "automate audio" in the settings?

When enabled, the audio of the M3 participant will be used instead of the host audio (see #1340) for details. We will likely remove this setting eventually.