search

perfanalytics / pose2sim

Markerless kinematics with any cameras — From 2D Pose estimation to 3D OpenSim motion
https://perfanalytics.github.io/pose2sim/
BSD 3-Clause "New" or "Revised" License
268 stars 48 forks source link

Suggestion: Add smoothnet #29

Open carlosedubarreto opened 1 year ago

carlosedubarreto commented 1 year ago

Hello. I just saw your commit with kalman filter. and I was thinking that maybe pose2sim could use smoothnet.

I tested it with a project that I'm doing using 4d humans, and it made some really good work. Not all the time, but most of the time the result improved a lot.

If you are interested, here is the link for the smoothnet code https://github.com/cure-lab/SmoothNet

carlosedubarreto commented 1 year ago

and the issue of 4d humand that I share the code I did and the results https://github.com/shubham-goel/4D-Humans/issues/33

davidpagnon commented 1 year ago

Thanks, this looks pretty impressive!

I had heard about DarkPose which processes 2D heatmaps to reduce jitter. It is good but older: Page: https://ilovepose.github.io/coco/ Code: https://github.com/ilovepose/DarkPose

Do you smooth 2D data, or the triangulated 3D coordinates directly (or the shapes)? I'll have to see how actually "Plug-and-play" it is, I don't have much time on my hands these days 😅

carlosedubarreto commented 1 year ago

@davidpagnon , I'm doing it applying directly on the rotation vector and translation. on translation somtimes it gives some odd results, but on the rotation vector it works very well all the times I tested.

I use on the final 3d data. the way I'm using now is from a single footage. Its outputing some great results, usign 4d humans. I'm not triangulating the data.

thanks for the darkpose link I'll check it out for sure.

About applying the smoothnet code, Iam almost sure that I mak a list with 3d vectors and throw that on the smooth script, than I save the result. Its basically it.

davidpagnon commented 1 year ago

I don't really have any rotations on my output until it is processed by OpenSim, only translations... I'll have a look for sure in a little while!

carlosedubarreto commented 1 year ago

it works with translations too.

And I was able to get joint translations from open sim, I just didnt know how to use them. I thought they were euler, because there was 3 values, but now, after suffering with another code, seems like opensim would be using rotvec... and scipy con convert it to quaternion or euler, or rotation matrix.

that is very important to use on things like armature in blender