search

mumble-voip / mumble

Mumble is an open-source, low-latency, high quality voice chat software.
https://www.mumble.info
Other
6.13k stars 1.1k forks source link

Multi-device support #6065

Open Krzmbrzl opened 1 year ago

Krzmbrzl commented 1 year ago

Context

Mumble uses certificates for user authentication. That's great, but causes issues as soon as one is trying to use multiple devices as per default each of them will use a different certificate and therefore these devices will be considered different users.

Of course one can copy the original certificate to the new device(s), but this solution is clearly suboptimal (from a usability as well as from a security point of view).

Description

It would be neat, if we were able to support multiple devices that are all associated with a single registered user on a given server.

This will require handling authentication with different certificates as well as a way to add and remove devices (certificates really) to/from a list of known devices.

As an extension, it'd be nice for a user to be online with different devices at the same time. In the UI these would display only as a single connected client but if that client is receiving audio that stream should either get sent to all currently connected devices or (if configured by the user) only to one specific device at a time.

Mumble component

Server

OS-specific?

No

Additional information

Related:

Krzmbrzl commented 1 year ago

Note: I will work on implementing this feature as soon as I have finished the database rewrite.

Hartmnt commented 1 year ago

As an extension, it'd be nice for a user to be online with different devices at the same time. In the UI these would display only as a single connected client but if that client is receiving audio that stream should either get sent to all currently connected devices or (if configured by the user) only to one specific device at a time.

I feel like this needs further consideration. I have to regularly deal with the following scenario: A user switches devices between desktop and mobile with two certificates. Ideally, I want the user to have the same registered ID on both devices, as in the "multi device support" you propose here. However, due to the extreme quality differences between Mumble desktop and the desktop microphone and Mumla and the mobile headset, I would really really also want to be able to set different, persistent local volume adjustments for both devices independently.

Krzmbrzl commented 1 year ago

I would really really also want to be able to set different, persistent local volume adjustments for both devices independently.

That seems to be impossible. Either clients consider these devices as the same registered user or not. Exposing the client's currently connected device(s) (which would be required as volume adjustments are handled on the client-side) does not seem like a good idea either. Using a hashed device-ID to share with other clients might solve the privacy issue but considering that a user can be online with different devices at the same time (once this feature is implemented) we'd have to send that hash with every single audio packet, which seems to incur too much overhead in terms of bandwidth to be worth it :eyes:

Hartmnt commented 1 year ago

In my scenario the clients would be still displayed as two different nodes in the tree, but both have the same registered ID, which is different from the client slot id, right?

I would rather have that than merging devices into the same visual client to be honest.

Hartmnt commented 1 year ago

The volume adjustment on the client side happens by certificate hash. The devices, all belonging to the same registered user, would still have a different hash. Just the same registered id. It should be possible to visually have two distinct clients for the same registered user with different certificate hashes.

My user-story only clashes with the last part of your proposal of joining the clients visually, if both are connected. A compromise would be to introduce different behavior only when both devices are connected simultaneously.

Krzmbrzl commented 1 year ago

In my scenario the clients would be still displayed as two different nodes in the tree, but both have the same registered ID, which is different from the client slot id, right?

If by client slot id you mean the client's session ID, then yes.

Not merging them in the UI seems rather pointless though. Why combine them in the first place then? Why treat them as the same user on the server but not in the UI? That seems inconsistent. :eyes:

The devices, all belonging to the same registered user, would still have a different hash.

That will have to be abstracted away as we really want it to be completely transparent what device a user is using. And this still doesn't solve the issue of having two (or more) devices online at the same time. How should the client know which adjustments to apply?

A compromise would be to introduce different behavior only when both devices are connected simultaneously.

How do you imagine things to work if only one device is connected? Registered users have a fixed name that gets assigned to them. So if you authenticate as that user, you'll get that name. And other than the name there isn't really anything that visually distinguishes clients :thinking:

Hartmnt commented 1 year ago

That will have to be abstracted away as we really want it to be completely transparent what device a user is using

So there is the requirement that there will be absolutely no indication to the client whatsoever that the user is using a different device? Not even a number from 0 to N?

Then this will be sadly pretty much impossible... Just please take into consideration that users might want different volume adjustments for different devices, but do not know that yet. :smiling_face_with_tear:

Krzmbrzl commented 1 year ago

Not even a number from 0 to N?

Something like that would be possible. Only the certificate hash would likely have to be abstracted in order for old features to remain functioning.

Hartmnt commented 1 year ago

Something like that would be possible. Only the certificate hash would likely have to be abstracted in order for old features to remain functioning.

If it is for backwards compatibility, you could still send the device hash as an additional field which older clients would simply ignore.

peacememories commented 2 months ago

Hi, I'm new to this discussion, but this is also a feature I would really like. And I'd like to speak in support of differentiating the clients to other participants. Ideally I'd even want the ability to join different rooms from different devices at the same time, although that is not a necessary feature.

Not merging them in the UI seems rather pointless though. Why combine them in the first place then? Why treat them as the same user on the server but not in the UI? That seems inconsistent. 👀

For me, the main point of combining the clients is a shared identity and, most importantly, shared permissions. This benefit still exists even if the clients can be distinguished by other parties.

I hadn't thought about the possibility of multiple volume levels per user, but now that it has been mentioned, that would be useful sometimes.

skobkin commented 2 months ago

I hadn't thought about the possibility of multiple volume levels per user, but now that it has been mentioned, that would be useful sometimes.

Volume levels are most likely being stored locally, so no synchronization would be possible unless specifically implemented in the server.

Hartmnt commented 2 months ago

Ideally I'd even want the ability to join different rooms from different devices at the same time, although that is not a necessary feature.

I think that is not even remotely what this issue is going for. The idea would be that your multiple devices behave as a single client, only capable of joining a single channel at the same time. If you switch channels on one device, you switch on all devices. My proposal here was that the devices would be displayed as individual "user indicators", such that different devices of the same user can have different client side properties (local volume, but also maybe local mute etc.). At no point was it intended to split those "dots", such that the different devices can join different channels.

Volume levels are most likely being stored locally, so no synchronization would be possible unless specifically implemented in the server.

Local volume levels are stored locally based on certificate hash. Always have been, and probably will be in the future.

If the last proposal of this issue is implemented as described, multiple vastly different devices (cellphone and PC for example) would be seen as the same client with the same certificate hash and same ID etc. While having some nice advantages, it would mean that you would NOT be able to set a different local volume adjustment per device.

And this will more often than not either mean that the client will quickly change between much too quite or - even worse - much too loud.

There was also some internal discussion about this a year ago:

Using network messages it should be pretty easy to tell the clients, which devices are currently connected per user. Making clients aware of the existence of multiple devices was deemed to not be the problem. The main problem here is how to address the UI ambiguity of "What happens if the user drags the local volume adjustment slider, if the target is connected with multiple devices"

We have noticed that separate user indicators are not strictly necessary to fix the local volume adjustment problem, if we get a little creative. Imagine we have implemented multiple device support, but there is only one user entry in the list for every user and we can not change that.

1) Store the local volume based on the set of devices that are currently connected. Instead of "volume of user0" we would have "volume of user0-device0-device1" and "volume of user0-device1" etc. as separate entries. When you adjust the local volume, you adjust it for the currently connected set of devices for the user.

2) Keep track of devices of a user separately "user0-device0", "user0-device1" etc. (not as a set) and adjust all currently connected devices when the slider is dragged.

3) Keep track of devices of a user separately "user0-device0", "user1-device1" etc. (not as a set) and track the last device the user was last talking from and adjust only that device

Krzmbrzl commented 1 month ago

We talked about how we could resolve the issue with volume adjustments on a per-device basis and what we came up with is something that we believe would be a general improvement to how Mumble works: https://github.com/mumble-voip/mumble/issues/6428

We believe that this feature should implicitly take care of all issues regarding different devices potentially having different mic configurations.