BrunoCunhaTeixeira
commented
1 year ago I was able to solve the problem. The calculated rotations are adapted to the original rig. Therefore, they can only be used with this rig. Mr. Holden was so kind and provided me with the original rig. I uploaded the solution on my github page: https://github.com/Nachbarino/PFNN-Integration-in-Unity
Hello Mr. Starke and other interested developers, currently I am trying to implement the PFNN in Unity as a practical part of my thesis. I am using your implementation for this. Firstly I would like to thank you for your great work and for making it available.
As mentioned in Issue 8, there are problems with the extraction of the rotations. Mr. Starke wrote at the time that he couldn't get the conversion from expmap format to quaternions because of the extra coordinate frame transformation between OpenGL/Maya and Unity. Could someone explain this in more detail? I don't understand which extra coordinate or frame is meant :/
So far I strongly followed the original implementation and wrote the following QuatExp method. Are there maybe errors in there?
In the original paper it is written that the rotations are calculated locally to the character root transform. I am a bit confused, because Holden et al. converts the rotations from the output to local rotations in their implementation (code line 1716). Are the rotations output by the NN global after all?
If I strictly follow the original version it looks like this:
I think the error lies in the use of the ForwardKinematics method, since this is not necessary in Unity because of the parent/child relationship. So I tried a little bit and used the global rotations from the output instead of the local rotations and multiplied it with the default rotation of the character joints. The results look better, but still not that good. I uploaded a Video of it on Imgur (link). On the video and on the pictures you can see two skeletons. The correct one is Mr. Starke's version and the second one is mine with the rotations.
Does anyone have an idea how to fix this? One of my guesses is that Unity and OpenGL use different coordinate systems and there are problems because of the handiness. For the rotations I use 4x4Matrix from Unity and convert it only in the last step to a quaternion when copying to the character with the mesh. Since I already tried to swap the rotations in different places, unfortunately without success. Does anyone have any other ideas how this could be solved, or even had similar problems during implementation?