Neos-Metaverse / NeosPublic

A public issue/wiki only repository for the NeosVR project
195 stars 9 forks source link

GetUserVoiceStream node/component #765

Closed H3BO3 closed 3 years ago

H3BO3 commented 4 years ago

A logix node that accepts a user and outputs the root slot of their currently equipped avatar.

Frooxius commented 4 years ago

Hmm this it a bit tricky, because there's not really a single root that the avatar can have, especially when it's equipped. Avatars can be composite, with some bits dynamically injected or equipped independently and pieces of the avatar can be parented under different parts of the user during equip, so giving a single root can be at very least a bit misleading.

What exactly is your use case here? It might need to wait for collections to handle this properly, as it would need to give you list of avatar objects.

H3BO3 commented 4 years ago

This is basically for the slot that you would point to for something like, say, the avatar equip node. The slot with the AvatarRoot component on it, isn't there only one of those? Right now you can do it if you use Get Object Root on a body node slot (typically hips). But I think there could be edge cases which might throw it off, hence the need for this node

LeonClement commented 4 years ago

Right now, one of the more reliable ways to get an avatar's slot, something I've found from one of ProbablePrime's setups, is to find the user's Grabber and get the ObjectRoot from there. But this requires the user to grab something which isn't ideal for a lot of scenarios.

Another was the one listed above by H3, which Shifty suggested.

If there was some way to distinguish an avatars slot reliably this would help future mechanisms to attach objects(Think of stuff like watches attaching to one of the proxy's within an avatar) or save an avatar's root slot for later use in projects.

TehTurk commented 4 years ago

You could just tie it into BodyNodeSlot in some form tbh. Typically you can just grab anyone from a user and then get object root from there.

H3BO3 commented 4 years ago

I'm still wondering what happens if it's a head and hands avatar, since they don't have all those body node slots. Need to run some more tests soon

TehTurk commented 4 years ago

In this case you'd have to specify the controllers instead I'd think. Tho I do know Heads and Hands do need to get standardized more with normal avatar conventions.

H3BO3 commented 4 years ago

Maybe the head node, since that's always consistent. Head and hands are pretty standardized already, I went through manual avatar creation with my new avatar, it's honestly fairly straightforward but tedious. You need the avatar root component in the top slot and a couple other things for convenience, then set up the appropriate tracking nodes. Everything else is basically supplementary so you can talk, have away visuals, etc. The reason full-IK avatars are so complicated is because there needs to be proxies for everything, so the IK can integrate with them. The IK takes over posing the actual rig/model based on the tracking data, but those tracking components can honestly go anywhere.

ProbablePrime commented 4 years ago

AvatarRoot is a component no? Just find that's slot and return that would work for most cases.

LeonClement commented 4 years ago

I'll give the component a try, I have no knowledge of it. I'm not really pushing for a new node if it can be resolved with something like a component. As long as its reliable is all I'd like.

H3BO3 commented 4 years ago

@LeonClement No, the AvatarRoot is a tagging component, specifying to the system that it is indeed an avatar. The proposed node would probably search for the slot this component is on.

Frooxius commented 4 years ago

That's kinda where the problem lies, there can technically be multiple AvatarRoot components or conversely no AvatarRoot components and the pieces under it can be moved while equipped.

It works for equipping, because that installs everything under AvatarRoot onto the user, but not the other way as there can be multiple installed avatar pieces.

Finding one object with AvatarRoot would technically work for most cases, but it's not a good solution long term, as it won't necessarily do what people would expect it to do in all cases and changing it would most likely be API breaking change. For that kind of task it might be better to wait for component access in general, which will let you enumerate all AvatarRoot components under the user.

H3BO3 commented 4 years ago

I'm still confused what you mean by "pieces". If an avatar doesn't have a single root slot with the avatar root, grabbable, etc. components on it, wouldn't it be extremely nonstandard and thus an edge case? Could you provide some examples where it wouldn't do what people expect it to?

Frooxius commented 4 years ago

The system works by filling in avatar "slots" with objects. Those can be equipped manually by the user, automatically from script or filled by a function if they're left unfilled. These slots are things like the head, hands, feet.

For example when you use a head avatar that doesn't include hands, those hands are auto-filled by a function, which essentially constructs two new avatar pieces (each hand) on the fly and then equips them. From user's perspective you just equipped a single avatar, but from technical you actually equipped three.

This could be done manually too, you can setup avatar that just equips the hands and keeps the head piece alone. This isn't really used widely (at least not right now), but can be done, so it needs to be considered.

What if you have multiple avatar roots in this case? Which one does get returned? What if avatar is equipped in pieces and there's no avatar root at all?

It's why I'm asking about the use-case for this, so I can better understand what the goal is and see what might be the best approach/solution.

H3BO3 commented 4 years ago

The most common use of swapping an avatar via logix is to get a reference to the user's voice stream, by storing the root to their current avatar, bumping them into another one that has the voice stream ref saved from it, then putting them back into the original. I believe this is the case where this conversation started, as dynamically getting voice streams was a security concern, but a workaround that works in most cases was found anyway.

However, there could be another solution to this, since most objects like this require it - A component that works alongside a grabbable, and sets the voice stream to a target field when it is grabbed? Much like the avatar voice stream setter, but one that works when grabbing instead. The intended use of this functionality is for gripped objects like microphones, walkie-talkies, phones, etc. so it would translate very well.

shiftyscales commented 4 years ago

I feel like that use-case could be taken care of via this issue, @Frooxius @H3BO3: #601

It's a deprecated component that is in need of a rework/redesign anyway.

Frooxius commented 4 years ago

Hmm, I don't really think it's good adding this node in particular for those usecases, since it's just to enable already hacky solutions to achieve a goal.

If getting the user voice stream is your goal, I can just simply add node to do that and add the appropriate security measures (mainly filtering the voice when in-whisper), since those problems would still be there even with the hacky approach.

That's the main problem too, if you wanted to use this to re-requip the user's avatar again, in come cases you'd just end up corrupting them and it would not be fixable issue, unless I changed the API of the node later on and broke existing stuff that uses it.

@shiftyscales That wouldn't help unfortunately, that only changes the active voice state, but doesn't get you the voice data.

shiftyscales commented 4 years ago

Accessing user voice streams could perhaps be something better for general component access/collections then to pull those references dynamically.

Similarly for the initial issue of getting the various avatar roots.

LeonClement commented 4 years ago

This can 100% wait for that. I think the request is that it exists in some shape or form, there's no need or demand for it as far as I know. We've got our hacky little solution to play with for now : )

Thanks for working things out and taking the time, I appreciate all of you <3

H3BO3 commented 4 years ago

I think Froox is saying accessing the voice stream wouldn't need collections. He could just add a node that does it in general, but filters it in whisper mode. That works for me

shiftyscales commented 3 years ago

Closing this issue out in favor of #915 which has more votes/relevant posts.