Join multiple channels simultaneously

[Solomute]

Problem: There are environments that require multiple "departments" to coordinate with each other, but they don't always have a hierarchical relationship with each other that lends itself to abusing channel linking and ACL. And besides, abusing channel linking and ACL is complicated and pushes the limits of what these features are intended to do in the first place.

As an example, let's look at a game I play called Artemis, a game where people work together as the bridge crew of a space ship, and if you have enough people you can crew multiple ships and have them all work together. Each ship wants to have their own channel for coordinating the operation of their own ship, but the captains of all the ships will also want a channel to coordinate fleet actions, the science officers will want a channel to coordinate how they want to split up scanning duties so that nobody wastes time scanning things that other people are already working on, the fighter pilots will want a channel to coordinate their attacks, and none of these people want to have to listen to each other. There's no way to do this in, really, any voip software out there short of extremely expensive matrix intercom solutions aimed at television broadcasters.

I can imagine other large scale games like EvE, ARMA, etc. run into similar issues.

Solution: Let the user join multiple channels at once. From a usability perspective I would create the notion of "channel slots," where you would right click on a channel and select "join in slot 2", "join in slot 3," etc, and bind push-to-talk and other functions to act on the additional slots. The normal channel switching method of double-clicking on channels would continue to work as it does now so that users can continue to use Mumble the same way they always have, but channels joined that way would be considered "slot 1."

[Solomute]

I've just learned that Ventrilo calls this concept "Phantoms" and seems pretty close to the concept that I'm talking about, except that Phantoms can't talk and you need to use shouts instead.

[mkrautz]

Thanks. It's great that you've provided your thoughts on how it'd work in practice.

[Natenom]

Isn't this already possible with shouting to channels|whisper to groups? At least thats what you can do for example for MMORPG (https://wiki.mumble.info/wiki/ACL_and_Groups/English#MMORPG_Raid)

But instead of sitting in several rooms at once every one has different shortcuts for shouting to other commanders, to their own staff members, etc...

If everyone uses different shortcuts for every special group then all these members are in "virtual" rooms together.

There is a game named "Project Reality" that does this with an own modified Mumble client with all the needed shortcuts. Can you check this if it fits your needs?

[mkrautz]

@Natenom

The problem as I see it is that you have to have a complicated ACL/link setup for this to work (to receive audio from the other channels).

If we just allowed users to clone themselves into multiple channels, it'd be a user choice, not an admin choice.

[Solomute]

The "MMORPG Raid" example requires a hierarchical channel structure, and it is not always possible to arrange the channels that people want to use in a hierarchical manner, nor should users be expected to have to come up with weird hacks in order to accomplish their needs. The fact that clever ACL hacks and workarounds are possible should not mean that those hacks and workarounds should be the way that users are expected to accomplish everything. ACL is a permissions system, it should only be expected to be used for channel membership authorization. People complain about the complexity of ACL not because the notion of channel permissions is confusing, but because of the complexity of the structures you need to build in order to use ACL to accomplish things that go beyond what it is suited for.

Also, the expectation that people use ACL to accomplish these sorts of things means that only people with admin permissions are able to set them up. If nobody with server admin is around, you don't have the tools to create any special communications structures you need. Ideally you shouldn't need admins to create structures for you, a system where a user can join multiple channels means that you can just hop on any Mumble server, have people join channels according to whatever communications structure they need, and get on with their game.

The solution of "just have everybody make a bunch of whisper lists" isn't really a solution. Again it's only suitable for things you can plan in advance. Situations where you might want to reorganize who is a member of what groups would be extremely impractical to manage. Not to mention that since the whisper lists exist on other people's clients, you cannot choose to join or leave a group at will, it requires everybody to update their lists for any membership change of the group, rather than that person simply being able to enter or leave a channel. Going back to my Artemis example, we'll typically play a half hour game, then shuffle everybody around between ships and crew positions and play another one. That means every single person needs to update their whisper and shout lists every half an hour. Or if traffic from one particular group gets to be too much and I want to mute it for a second, I can't, because again that traffic is coming from other people's whisper lists and not from a channel that I can mute or leave.

Project Reality's Mumble client appears to be integrated into the game rather than available as a client for general purpose use.

You seem to be taking the attitude that users should adapt their workflow to the software, rather than the software catering to the user's requirements. I really shouldn't have to say that this is a horrible design philosophy. The entire point of software is to cater to the needs of its users. If there is an architectural reason such as a great deal of legacy code built under the assumption that you will only be a member of one channel at a time, and attempting to change this would break a lot of things and involve a lot of refactoring, that's a totally valid reason to reject this feature request. "There is very little demand for this so it is a low priority" is another valid reason (though I believe that if this feature makes it in, people will very quickly find demand for it). But I don't see "you can already sort of do this if you're willing to do a lot of manual labor" as a valid reason.

I think the best solution from a usability and ease of implementation perspective would be Ventrilo's "phantoms" concept (right click a channel, "join as phantom," and now you are listening to that channel), plus the existing "shout" functionality and some additions to the overlay feature so that you can tell where whatever voice you're hearing is coming from.

[Solomute]

I come from a background in television broadcasting where we have some pretty heavy intercom needs, to say the least. The most common concept used in heavy duty broadcast environments is known the "matrix intercom." There's a pretty good book about broadcast intercoms here, there's a lot of nitty gritty technical stuff in there but there's also some great high level explanations of intercom systems engineering that I think is pretty worthwhile for anybody developing any kind of large scale communications solution.

I imagine the Murmur server isn't doing any audio mixing, just routing Opus streams between users? If so, it's pretty much already acting like a matrix intercom under the hood, everything else is "just" user interface (scare quotes because yes, I'm aware that UI is actually the hard part.) But I think that just being mindful of the fact that Murmur is just a rules engine for deciding where that Opus stream gets routed to, and that things like "channels" and "whispers" are just ways that we present that to users and are not structurally important to how Mumble works, can open up a lot of flexibility in how we empower users to communicate.

[mkrautz]

I agree that this feature would be desirable.

[Chris2000SP]

maybe: mumble -m

Edit: https://wiki.natenom.de/mumble/benutzerhandbuch/murmur/acl/egoshooter-kanal https://wiki.natenom.de/mumble/benutzerhandbuch/murmur/acl/mmorpg-kanal

This is a ACL list that can do this. Own channels for people who want to talk with they own group, and if they want to talk with others push the whisper button. AND you can change the channel without changing the whisper button settings.

If you have problems let me know.

Edit: I have to add this to the mumble.info wiki

[Solomute]

Again, this only works with hierarchical channel structures and requires admins to implement the necessary ACL. Did you read the thread at all, or...?

[Chris2000SP]

Yes i have read the whole thread. And if you don't have admin rights to do that in your own channel then you have to think about the right Server for you.

You can also make your own server, because mumble is free software!

Edit: For reference the ACL and Groups are able to give every one on a server who is registered control every channel admin rights of every user who wants his own channel. And mumble has no limit to make subchannels depth to make a big tree of channels

Edit: And the Server Administrator don't lose control if he makes a channel area for the users who want to admin they own channels. The Server Admin is the one user is listet in the admin group in the root channel. Below that root channel there be possible to make USER channels with the users admin rights. You have to think about inherit of rights in the subchannel depth, and the possibilities of every RIGHT you can use.

[mkrautz]

@Chris2000SP Please. @Solomute made a feature request, exactly to avoid having to deal with admins and ACLs in this scenario.

I don't think there is anything wrong with the feature request, and I am sure @Solomute is aware that it is "possible" to do something similar with ACLs/shouting.

And please. Do not tell people to go make their own Mumble server in feature requests. That comes across as very rude.

[Chris2000SP]

I have a problem with that "ACL is complicated". I have made this manual for exactly this scenario and i have rewritten it by make it easier. So why is that a problem?

[Fahrradkette]

Thanks @Solomute for bringing it up. Would you mind sketching the UI on paper? If I understood you right, its basically right-clicking on a channel to add/remove whisper list targets?

@mkrautz do you think it should be possible to implement this (without attenuation) using the existing whisper list (ShortcutTarget) infrastructure, i.e. would it only require UI/client side modification? I'm afraid of breaking compatibility.

feature creep

My experience with that feature in ProjectReality is quite good, except for that it was impossible to attenuate certain "channels" (or whisper lists actually).

For instance, when under attack, you want to listen to your ship crew/team/squad, but during "relaxed times" you want to hear the other officers louder i.e. attenuate the team channel when an officer is speaking.

This needs at least some additional settings UI where you set keys to switch presets containing those channel priorities. I also don't know yet how the existing attnuation/priority speaker infrastructure could be used.

[Solomute]

If I were designing the UI, I'd add an item to the context menu when you right-click on a channel, called "Listen to channel":

The server should only allow you to do this if you have ACL listen permissions. The server should kick you out of the channel if permissions are changed and you no longer have listen permissions. "User joined your channel" sound effects and notifications should play when you start listening to a channel.

There should be a server setting to limit the number of channels one person can listen to, so that they can't DOS the server by joining all the channels.

While you are listening to a channel, your name is displayed in that channel in italics with an ear next to it, to indicate to people in that channel that you are listening:

This is important because people always need to know who is listening to them, so they don't say things that they want to keep private!

If you want to stop listening to a channel, you right click on it, and the context menu item now says "Stop Listening":

If you want to talk to that channel, you bind a key to shout to that channel. Your normal push-to-talk button talks to the channel that you joined normally. There should be a way to bind a key to listen/stop listening to a channel, just like you can currently bind joining a channel.

This also interacts with the overlay system, if someone talks in a channel you are listening to it should show up in the overlay with a label showing which channel you are hearing it from.

The note about needing to hear certain channels louder than others is a valid one. The best thing I can think of is:

Add an option, "dim" to the context menu when you right click a channel. Make it possible to bind a keyboard shortcut to toggle dimming a channel. Dimming a channel makes it quieter. Add an option to the audio options menu, "dim amount", where you can set an amount, in dB, that you want dimmed channels to be quieter by.

I would suggest that further discussion on that feature go in a new feature request, if listening to multiple channels is ever implemented.

[Fahrradkette]

Thanks a lot for your thoughts and screen shots.

What about the concept of a "meta channel" (for lack of a better word), a special channel type which acts as a "shout target list". Joining (or subscribing to) that channel requires you to assign a hotkey for it. Those channels would act as "engineers channel", "commanders channel" etc.

That also means that all the users in that channel would have their names in italics...guess having the channel itself in italics would be better (if italics isn't already used to indicate stuff like temporary channels).

I'm still trying to think of an implementation which uses the existing "shout to channel" functionality so it's less likely to break compatibility with older versions and less code to touch :)

Edit: I think that requires to have channels having an additional flag "isMetaChannel" which existing servers don't understand. Having the need of rolling out an update to 1.2 installations sounds bad to me.

[Chris2000SP]

I won't get rude on this, but you can also simply Link channels and get the ACL @out 'speak' deny on the opposite channel. Or you can put that rule on all channels and get a whisper to channel key.

[LJAndersson]

That does not work in my case, simplified but not exaggerating:

10+ channels (actually 10 groups of linked channels) with 250 users each where each user may or may not need to hear what the is said in the other channel. For obvious reasons though we dont want to have 2500 people in the same channel and we cant ask the "commanders" to set up 10*250 (potential) whisper combinations each time

[SM0VXI]

Like @Solomute my background is also in the broadcast industry. I have been using Mumble for some time now as wireless intercom in a test environment. So I'm going to take the opportunity to air my thoughts on the subject of "simultaneously joining multiple channels", since it is the only real fundamental feature I'm missing in my use case.

I understand that big overhauls to (G)UI might upset current users and is not desirable. So I agree with @Fahrradkette 's approach to implement an additional channel type. Like @Solomute describes, the essential difference to an "ordinary" channel is:

• the user doesn't "join" a channel. Instead the user "listen" to the channel, thereby monitoring the activity in the channel. The user can chose to listen to one or many channels at the same time. The user has the possibility to assign a "listen" key for each channel, so s/he instantly can toggle between listening and not listening from keyboard.

• the user-list of a channel indicates who is listening at that moment.

• the user can assign a "talk" key for each channel. The behaviour of the key can be either PTT or Latching (toggle on/off).

I also want to add the following:

• being able to adjust the audio level from each channel is desirable. Perhaps by using +- keys to adjust the level of the currently highlighted channel.

• prioritising would be desirable: if there is audio on a channel with a higher priority, the audio of channels with lower priority should be dimmed. Audio of channels with the same or higher priority are not dimmed.

• a parameter to set the dim level (described above) is desirable.

• not having to receive more than one stream from the murmur server is desirable: mixing of audio from the concurrently monitored channels must then take place on the murmur server, and being delivered as one opus stream to the mumble client. This also implicates that the adjustment of audio level from each channel has to take place on the server (see bullet point about audio level above).

I'm curious if someone could elaborate around the work on the server-side that would have to be done to accomplish this?

[Solomute]

Based on my limited knowledge of the issues, I believe that having the server actually mix audio may not be desirable. 1) It would be a major architectural change to add an audio mixing engine that all audio has to pass through, which may not be justifiable for just one feature. 2) The CPU overhead of adding a decode+mix+encode to every transmission between every user would make hosting a mumble server far more expensive, and probably prohibitive. I don't think it would be possible to do 1000 slot servers, which are currently common. 3) Adding an extra decode/encode step would add latency as well as generational encoding loss, both of which are undesirable.

This is not a huge deal in our business of show production where we probably run our own server on our own LAN, but the majority of mumble users are buying hosting by the slot from hosting providers on the internet so their needs and the needs of hosting providers are probably most important.

[SM0VXI]

I have no facts on this, but there must already exist a decode/encode stage and mix engine in the server? Lets say that two different users are talking at the same time in the same channel, then the rest of the users in that channel will hear both of them mixed. So as far as I can understand input mixing must already take place server-side. It should be possible to achieve the necessary output mixing by the same means.

As you say it would add to CPU load. But the only other approach that I can come to think of, is sending a stream of every channel that the user has chosen to monitor. In your opinion, would the increase in network load this will cause, not only server-side but also client-side, be a negligible problem compared to increased CPU load at the server?

I did a quick and dirty survey of a few commercially available systems for telecom, military and broadcast. They all take care of mixing server-side (except for self-monitoring/sidetone for delay reasons). But then again, most of them have big hardware budgets and don't have to rely on hosting providers.

[Qaplop]

Is anybody finally working on this feature request? There is still public demand for the feature.

I would like to see it exactly as described by Solomute. Thanks for your proposal mate!

[wilssonbrothers]

I would also like to see it exactly as described by Solomute. It would be great to get this because Mumble is becoming more and more common in broadcast (TV production). http://www.sienna-tv.com/ndi/intercomserver.html

[Krzmbrzl]

For all I know, there is currently noone working on this feature. We are a very small Dev-Team (currently only 2 active devs). And I guess there have been more pressing issues that had to be dealt with.

I am encouraging anyone willing to contribute to give this a shot though. Simply open a PR or get in touch with us via the official IRC channel (preferrably through Matrix) and we'll try our best to guide you through the contribution :)

[matthiasgraf]

Sort of in line with the above I'm also using Mumble in a TV broadcast setting.

I'm currently using it to do small local intercom setups between remote contributors in place of costly comms setups like RTS and Clearcom. I've assigned a number of keyboard shortcuts to a little controller to do something similar to the above scenarios using ACL's, so various shouts and various rooms links so 1 user can speak to or listen to multiple other rooms/users. The controller has lights that can toggle on and off, so if I'm linked to room A (listening), the corresponding button on the controller lights up and when I'm not linked it turns off.

The issue I'm running into is the controller currently just toggles the lights statically when you press the key, it doesn't poll Mumble so they can get out of sync pretty easily (Mumble seems to miss commands decently often). I'm trying to figure out if there is a way to get the status of room link/unlinks using an API so that I can then send that information to my controller to dynamically control the lights so that they reflect the true state of Mumble and not what my last button press was (whether the command took or not), or, to actually control the links/unlinks to the same effect. Any help would be greatly appreciated!

[seantalts]

I might be able to take a crack at this; do any of the current devs have an outline for what parts of the code would need to change? I'm new to the source code though not new to Mumble (thanks for developing it!).

[Krzmbrzl]

I might be able to take a crack at this

That'd be awesome :)

outline for what parts of the code would need to change

That strongly depends on how you want to implement this feature.

From reading through this issue I have the feeling that what really is desired is a way to eavesdrop to other channels. For that I think the suggestion with "throwing in ones ear" is a good approach to this. So if you want to do it that way, I think these are the steps necessary:

1. Extend the Mumble protocol so that it can be communicated that a user wants to eavesdrop (or stop doing it) to a certain channel. For normal channel joins the UserState message is being used so I think extending that message type with the respective field(s) might be best. https://github.com/mumble-voip/mumble/blob/f8ee53688353c8f5e1650504a961ee582ac16668/src/Mumble.proto#L174
2. Extend the User class to not only hold the channel a user is in but also which channels s/he wants to listen to additionally. https://github.com/mumble-voip/mumble/blob/f8ee53688353c8f5e1650504a961ee582ac16668/src/User.h#L33
3. Extend the channel class in a similar way. https://github.com/mumble-voip/mumble/blob/f8ee53688353c8f5e1650504a961ee582ac16668/src/Channel.h#L41
4. Adjust the Server-code that is responsible for routing incoming audio packets to the clients to also including listening clients. https://github.com/mumble-voip/mumble/blob/f8ee53688353c8f5e1650504a961ee582ac16668/src/murmur/Server.cpp#L1082-L1085
5. Design and implement the UI for this feature. For this you will probably have to fiddle with src/mumble/MainWindow and src/mumble/UserModel

When fiddling around with the protocol you'll also have to edit the Messages.cpp files in both src/umble and src/murmur to handle the incoming messages appropriately. Furthermore you'll also have to edit src/mumble/ServerHandler in order to implement a mechanic for a client to request changing his/her listening state for the different channels - that means you'll have to add a method that'll create and send the proper UserState message for the server.

If you are going to give this a shot I suggest opening a WIP pull request early on so we can directly start commenting on the code and giving hints :)