Explore additional models for integrating camera feeds more naturally into a Hubs experience

[misslivirose]

Is your feature request related to a problem? Please describe. We get feedback that for some set of use cases, it's desirable to add additional emotional context to a Hubs space by sharing webcam feeds from one or more users within a space. While Hubs currently has the ability to share webcams as a 2D media panel within a Hubs room, the feature is minimal and not well-integrated into the workflows and interface that we currently support.

Thesis While spatial meeting experiences provide a shared environmental context, current avatar-facilitated communication models in Hubs are lacking in a shared emotional context. Improving shared emotional context within a virtual space will make team meetings more meaningful and representative of the individuals who are present

Describe the solution you'd like This work would have several potential components. This should be viewed as a top-level spec to explore and come up with a more concrete proposal and plan of implementation. The feature work to support explorations into the thesis above (specifically focusing on webcam features - more detail on this below in Describe alternatives you've considered) may include improvements in the following areas:

• Improve overall camera playback in Hubs so that it is performant to have a camera feed from each avatar in a room (up to 20 users).
• Create a ‘camera feed’ avatar. When a room visitor comes to a space wearing this avatar and turns on their webcam, they are prompted to either place their webcam in the room or wear it so that the camera faces outwards and follows the user around.
• Create a 2D panel where camera feeds appear (similar to Zoom’s interface) in the client.
• Test different audio capabilities for an avatar-video hybrid approach.

Example scenarios Meetings where a presenter wants to be on camera; more emotional-driven meetings (e.g. professional 1:1 meetings, team building exercises), events between friends and family members

Describe alternatives you've considered Alternative models to the thesis around supporting more shared emotional context could also include improvements to avatar systems directly. That is out of scope for what should be considered in this issue, and instead should be considered as a separate solution space in parallel to the camera improvements.

Related

2327

2735

2610

2660

Status / Action Items Add as consideration for H1 2021 feature work; no immediate action required.

┆Issue is synchronized with this Jira Task

[misslivirose]

@mbransn @heaversm - This isn't something that we have concretely scheduled but may be an area of exploration for how people may want to choose to represent themselves in different contexts. Basically, same thoughts as #3285 :-)

[misslivirose]

We will be addressing this in two separate ways, which may share some design components. Based on work from our hack week, we plan on releasing two additional forms of webcams in Hubs:

1) A '2D dock' of videos that can sit on top of the Hubs client to replicate video conferencing setups. 2) The ability to wear an avatar that has a space to display a camera feed and move it around in the world the way that other avatars do. (#2327)

Additionally, we will still also have the ability to surface web cams as 2D planes within the Hubs room (for example, placed in a media frame).

A version 1 prototype of this might incorporate the hack week work into a consolidated design for selecting where a camera feed should go from DPX. We may want to keep that under a feature flag until we have also established our requirements for monitoring performance of webcam feeds and other technical considerations that would arise from that.

Version 2 of this would include filters that can be shared over the video, including the chromakey/green screen request in #2660