Holographic Voice Chat/Call

mpmxyz commented 1 year ago

Is your feature request related to a problem? Please describe.

Imagine calling someone over to 'your' world or giving someone quick advice. Of course you can use text/voice messages. But pictures tell more than a thousand words.

So you can personally go to them and invite them over. (loading their whole world before locating them within as a side effect)

But now imagine that you are living in the future...

Relevant issues

No response

Describe the solution you'd like

One could "call" contacts on their friends list. If answered you would not only be able to talk but see the other persons avatar in front of you or over your wrist. Internally I could imagine this as a bare-bones "overlay" world where only the default avatar of the other user is spawned. That would make initiating it relatively lightweight while allowing custom visuals to be displayed.

This approach does not make it possible to see the environment the user is in. (keeping privacy intact) But it may be desirable to show/share objects during the call. Maybe it is a good idea to grab an object and use a context menu? ("move to call"/"take from call"/"send")

Describe alternatives you've considered

It is not a 'must-have' feature but one that could be quite eye catching to outsiders if done well.

Additional context

No response

shiftyscales commented 1 year ago

This sounds related to #3274 which would be a fairly substantial body of work.

JackTheFoxOtter commented 1 year ago

Major feature but insanely awesome. This would be extremely useful and honestly a really cool way to introduce more cross-world social features. Since this would internally be handled like any other world, it shouldn't be harder to expand this to allow group calls, although this could become an interesting UI challenge. Also custom ringtones per contact! :D

FlameSoulis commented 1 year ago

The issues will be having a size limit on the objects/avatars that get used as part of the message. Assuming that could be a cloud reference (perhaps required?), you would have the following criteria:

Avatar IK data (or at least the tracking points to be manipulated, which includes rotation and positions based on a specified recording frequency (30/45/60/120 FPS). Higher frequencies means higher data requirements.
Message Audio. This is already baseline.

Additionally, you would need to know where to spawn the 'message' so it can be reviewed.

Quite a few challenges. It is doable, primarily if based on an inherited voice message internal class. Still, it also brings to question how much data the extra points would take and how the infrastructure for it should be calculated...

mpmxyz commented 1 year ago

The issues will be having a size limit on the objects/avatars that get used as part of the message. Assuming that could be a cloud reference (perhaps required?), you would have the following criteria:
* Avatar IK data (or at least the tracking points to be manipulated, which includes rotation and positions based on a specified recording frequency (30/45/60/120 FPS). Higher frequencies means higher data requirements.

* Message Audio. This is already baseline.
Additionally, you would need to know where to spawn the 'message' so it can be reviewed.

Quite a few challenges. It is doable, primarily if based on an inherited voice message internal class. Still, it also brings to question how much data the extra points would take and how the infrastructure for it should be calculated...

I think there is a slight misunderstanding: This suggestion is about a live call between 2 or more users, not an asynchronous message that you can play back at any time. It would be closer to a secondary world overlayed on top of the actual world you are in than a voice message recording.

FlameSoulis commented 1 year ago

I think there is a slight misunderstanding: This suggestion is about a live call between 2 or more users, not an asynchronous message that you can play back at any time. It would be closer to a secondary world overlayed on top of the actual world you are in than a voice message recording.

Ah okay. This might be able to be prototyped in a way to see how well it goes.

Neos-Metaverse / NeosPublic