Hubs-Foundation / hubs

Duck-themed multi-user virtual spaces in WebVR. Built with A-Frame.
https://hubsfoundation.org
Mozilla Public License 2.0
2.13k stars 1.41k forks source link

Facilitate a mode where a user can participate in a session with a single avatar across devices #1690

Open misslivirose opened 5 years ago

misslivirose commented 5 years ago

Is your feature request related to a problem? Please describe. It is difficult (or impossible) for some types of sharing / interactions to be controlled from inside of a VR headset, so users often have to either have two avatars in the room at the same time, one per device, or they end up not bothering with the VR headset at all.

Describe the solution you'd like "One of the reasons I would love to have a "I connect to the same avatar via multiple machines" is that I'd love to be in-VR in a standalone HMD, then pop to my desk for a while to screen share a presentation or take notes, then go back to VR, all of them controlling the same avatar" - per @blairmacintyre

The ability to have one avatar that is driven by multiple client sessions (likely through a Hubs account) would make transitioning between devices more effective, granting a wider range of flexibility in how Hubs can be used.

gfodor commented 5 years ago

i guess i'd like to know more about what breaks down specifically if you have two sessions open. is it because you want to have a single visual representation? is it because it's awkward to deal with the fact that you are in a different position when you context switch? if we tease out the core issues it may result in some minor changes we can make to the "multiple session" experience to make it less painful.

blairmacintyre commented 5 years ago

I think it's both of these. The use I've encountered is

Having two avatars in the room means I need to control them both separately. I don't want both "near each other" because that's ugly and confusing. I don't want to have to remember to manually mute one of them. If I want to drop something in the room, and pin it, I'd like it to stay if one of them leaves (I don't know if the pinning recognizes they are both on the same account and does this already).

What I would like is hubs to recognize I'm connected multiple times (perhaps this is an option presented when the same account connects multiple times), and somehow let the avatar be controlled by one and only one of the displays.

Perhaps a simple hack might be that it actually has multiple avatars internally, but only shows one. On the screens for the "hidden" (not currently in control) avatars it shows a bit of UI somewhere that says "avatar controled by device X, click to grab control" and internally it just sync's the position/orientation of the avatars and shows the one for that device, hiding the other. One key would be switching audio (in/out) when you switch devices, and of course the representation of the avatar (hands / no hands, perhaps even the name and appearance so I could choose to indicate what I'm doing in the room to others by variations in my name/avatar).

But, of course, I don't know what other tricky things related to object ownership.

The keys for me are that

gfodor commented 5 years ago

cool thanks for that additional context. this is a tricky one. one challenge with hiding avatars is that it introduces asymmetry, which could be an abuse vector. (eg, you use this feature to hide and then listen in on others' conversations from across the room.) this is part of the reason the "hide" tool has a server-enforced, bi-directional cutoff of all data.

it's hard to come up with a clean solution that solves the use case perfectly. I think a model where we get to the point where the desktop + tethered experience (you can view/interact on the screen through the eyes of the headset, and use the menus) matches the others could be an incremental improvement. for example, when you use a link code from a PC to move the room to standalone headset, we could then set the originating PC to join the room and pin the camera to the standalone's tracking data (we'd likely do this by hiding the avatar as you suggest and just syncing the transform to the other session, but it'd ensure that there's no visual/audio disconnect between the two sessions.) the downside of this approach is you wouldn't have separate agency on the PC wrt the camera (similar to the tethered case.) it's complex enough to implement it may not be worth it vs some other approach.

blairmacintyre commented 5 years ago

I think I was essentially suggesting what you are saying: synchronized the pose of all "replicas", and when you switch devices, show the correct one and hide the others. That's essentially the behavior I want, but I (obviously) have no idea how hard it is.

gfodor commented 5 years ago

ah gotcha, I thought you were referring to having multiple vantage points (one per device) -- i agree that if we unified the camera across the devices that mitigates the social system stuff and seems somewhat tractable. the main unknown thing is if we had any modality to it that required understanding which device you were "using", that could be prone to error.

blairmacintyre commented 5 years ago

yes, yes. Those things, and if there are "ownership" issues with content. But, I was imagining that it would be tied to an account (so instead of logging you out if you logged in multiple times, it would offer to "join" them, as one idea), so perhaps ownership just "comes out in the wash".

I would imagine a UI like on Windows MR (where is shows the "use Windows-Y to regain cursor control" across the top). One screen would indicate it's live; the others would have some indication (probably in the area where the mic mute, etc., are) that this one isn't "active" (the mic would be grey'd out, and some UI showing that it's a slave) and a UI there allowing you to "grab control" on this one.

machenmusik commented 5 years ago

Thinking of no-code low-tech ways to do this... would using a product that can switch between virtual desktop and in-VR modes suffice?

emclaren commented 4 years ago

received feedback from a community member that they found it confusing that there were sometimes two avatars for the same person in the space. They mentioned that they thought about "hiding" one from view, but weren't sure if that would hide both of the avatars for that person