Fullbody constraints and freeing the head proxy

[alexderpyfox]

Is your feature request related to a problem? Please describe.

So basically i made a comparison video between Neos and vrchat way of doing fullbody tracking and found out the issues that it has in Neos https://youtu.be/2ZsiUyq7Jfc vrchat https://youtu.be/KzJYVMAx8Io Neos So basically in Neos the head locks out at the top causing the bones down the hiarchy to fold in on themselves also in vrchat the bones above the hip ie the spine and chest and neck seem to have rotation constraints to it you mostly only rotate in the x axis so when rotating the hip only the hip should rotate and nothing else so when the hip shifts up the whole avatar moves so it seems the hip as priority is a good idea unlocking the headset to be able to also move up to prevent scrunching and adding rotation constraints to spine chest neck

Relevant issues

Issue 1120# but mine is more about the solution

Describe the solution you'd like

It to be added or a possibility of us to mess around with constraints and the head proxy movement ourself

Describe alternatives you've considered

Modded client but prime said as i understand that it is not possible for ik fixes because of the way it syncs across to other users

Additional context

😔

[Frooxius]

I'm sorry, but I don't quite understand what you're actually asking for.

The IK in Neos doesn't have constraints like that that could be unlocked. It's mostly set of solver passes that produce the behaviors.

The head proxy is just an input that tells the system where your head is so it knows where to position the avatar head. You can move it if you're not wearing the avatar, or move the target, but then it won't match the position of your head.

[alexderpyfox]

I have more info here about 3 specific issues with fbt it's in a Neos folder incl video and mocap data neosrec:///U-alex-derpy-fox/R-ff2c5e08-48ae-4ee2-bd42-70d40f6838e1 the mocap is one recording but in the video it specifies the 3 issues issue 1 head rotation also rotates hip issue 2 scrunching issue 3 moving the hip down in the video I talk more indebt about the 3 specific issues i recorded the mocap alongside it i also can divide the mocap in 3 seperate pieces if you want but the video is important to see what each issue is edit: oh in the end i say the view becomes misaligned because of the "hip rotation" but I meant hip position

[alexderpyfox]

My possible solution for issue number 3 is basically with the fullbody calibrator also have a view position adjuster orb like the ones for the feet and the hip so when you drag the hip down you then could move the view point to be at the same place of the models head Or as a solution but not sure if this is possible having the viewpoint be attached to the head of the model so it will not get seperated from the body in any way when moving in fbt

[alexderpyfox]

Should I record more data or is the one I put in the folder sufficient?

[Earthmark]

I'm trying to understand what your saying so this can be more easily parsed by the team, but I am slightly confused myself. Would you be willing to work with me so I can understand and format this request to be easier to parse?

I think much of my difficulty is trying to parse your video, but I think these are the three issues you are describing?:

1: Rotating the hip tracker causes undesired neck or head contortion VRChat Neos In VRChat the avatar lifts, but retains the neck and head orientation. In Neos the avatar head attempts to not move as much as possible.

2: When tilting the head, the neck in Neos moves much farther than expected. VRChat Neos I think this is a different form of 1, but the users headset is moving downward in Neos.

3: I think this is the same as issue 2, I could not see a new request in the video, please advise.

a: Brought up being able to see the inside of your head VRChat Neos This was not brought up in your list, but it was brought up in both videos.

Some things that may help in the future: Please have the camera as somewhat stationary in the world in Neos, I'm getting quite motion sick trying to watch the Neos video. Using on screen text or clearly making marks or words in the video so the two can be aligned would make it much easier to try to understand what data points you are comparing. I do not believe I got the alignment right for the videos.

[alexderpyfox]

Yeah I could show you in-game are you on ATM so we can record it better together 😅

[Earthmark]

I had misread the issues described.

Issue 1 neck scrunching - When looking up and down the avatars neck collapses in undesired ways. On the Rex this is particularly noticeable as the feathers 'flick' up and down.

Issue 2 is that the avatars head can move independently of the users viewpoint. The expectation is the avatar's head remains completely fixed to the headset, any movement of the avatar head away from that is undesired.

This often happens when looking up and down after dropping the avatars hip to compensate for back slouching and neck scrunching.

Issue 3 is that when looking up and down, the avatar's hip appear to roll forward and backwards. This is the 'slouching' behavior, where the back of the avatar kicks outwards as it is pinched between the head and hip. This leads to the hip appearing to rotate due to head movement which is undesired. It is expected that the hip and lower chest do not rotate a significant amount due to head rotation. Some slouching may be OK, but the current amount is undesirable.

Issue 4 is covered by @Ducky07 in #1120, when completing calibration the hip shifts upwards and becomes misaligned from where it was assigned.

The goal of this request is trying to provide the ability to change the way the hip tracker is used in full body tracking, this option would provide the flexibility to solves all four cases.

Currently it appears the headset and hip tracker both act as fixed points. the goal would be to change the hip tracker from being the actual avatar hip, to being a point where the hip aims to be and the avatar hip can translate up and down based on the headset position and avatar's back length (neck, back, spine...).

This resolves issue 1, neck 'scrunching', as the back will remain stiff, and the neck will always have vertical space as it reserves that space in the back length calculation

This also resolves issue 2, the avatars head moving independently, as the headset and head are fixed, and the rest of the avatar is solved based on the head position.

This also resolves issue 3, looking up and down changing the lower chest rotation, as the deflection of the spine can be more easily controlled, as it is no longer pinched between two fixed points.

Note: VRChat does this in the other direction, their hip is an anchor and the head can translate vertically, however as Neos renders the head of the avatar around the viewpoint that would make issue 2 worse.

Examples and video to be added by @alexderpyfox, some issues are cropping up in neos with the recording tool.