After running on a 3-second squat video I got the following result
Basically the problem is that the skeleton is leaning forward, this is the result of using ax.view_init(elev=12., azim=200) in motion2video_3d function.
Additionally, the quads seem to be compressed (shorter length) at the bottom of the squat. I will test this on a longer version tomorrow with a slightly side angle but I don't believe that's the issue. Is this a common problem? Does it come from the training dataset that does not this kind of situation? Or is this a limitation with MotionBert potentially? Can I get an explanation?
When I run my own visualization script, the limb length seem to be fine
but ya as you can see the coordinates are messed up and this is because my coordinates mapping look like this
`lines = [ax.plot([], [], [], lw=2, marker='o', markersize=6, markerfacecolor='w')[0] for _ in range(len(joint_pairs))]
def update(frame):
ax.collections.clear() # Clears previous frame dots
j3d = motion[:,:,frame]
for i, line in enumerate(lines):
limb = joint_pairs[i]
xs, ys, zs = [np.array([j3d[limb[0], j], j3d[limb[1], j]]) for j in range(3)]
line.set_data(xs, ys)
line.set_3d_properties(zs)
if limb in joint_pairs_left:
line.set_color(color_left)
elif limb in joint_pairs_right:
line.set_color(color_right)
else:
line.set_color(color_mid)
return lines
ani = FuncAnimation(fig, update, frames=motion.shape[2], blit=False, interval=50)
plt.show()`
while vismo.py plots ax.plot(-xs, -zs, -ys, ... I am a bit confused about the order and why this changes limb length
Hi this is fantastic work!
After running on a 3-second squat video I got the following result
Basically the problem is that the skeleton is leaning forward, this is the result of using
ax.view_init(elev=12., azim=200)
in motion2video_3d function. Additionally, the quads seem to be compressed (shorter length) at the bottom of the squat. I will test this on a longer version tomorrow with a slightly side angle but I don't believe that's the issue. Is this a common problem? Does it come from the training dataset that does not this kind of situation? Or is this a limitation with MotionBert potentially? Can I get an explanation?When I run my own visualization script, the limb length seem to be fine
but ya as you can see the coordinates are messed up and this is because my coordinates mapping look like this `lines = [ax.plot([], [], [], lw=2, marker='o', markersize=6, markerfacecolor='w')[0] for _ in range(len(joint_pairs))]
while vismo.py plots
ax.plot(-xs, -zs, -ys, ...
I am a bit confused about the order and why this changes limb length