search

how4rd / meshflow

An implementation of the MeshFlow video stabilization algorithm by S. Liu et al.
31 stars 8 forks source link

About the adaptive weight $\lambda_{t}$ #6

Closed Nyquist1011 closed 2 years ago

Nyquist1011 commented 2 years ago

Thanks for your great implementation, it's so great!

When I read your code of the calculation of the adaptive weights $\lambda_{t}$, I found something baffles me.

In the function "_get_adaptive_weights", the translation element $T_v$ should be normalized by the image width and height and the function get these two params from here in the function "_get_stabilized_vertex_displacements".

However, the shape[0] and shape[1] of "vertex_unstabilized_displacements_by_frame_index[0]" may represent the row and column count of the meshes rather than the height or width of the original frames. Besides, it seems that the input is in the order of "frame_height, frame_width" while in the definition of the function, it's "frame_width, frame height".

I wonder if there are any minor problems here. Thank you!

how4rd commented 2 years ago

I'm glad you like the implementation!

I agree, it looks like you found two mistakes: using the mesh rows/columns instead of the frame height/width, and swapping frame_width and frame_height in the call to _get_jacobi_method_input. Both those mistakes should only affect runs that used the ADAPTIVE_WEIGHTS_DEFINITION_ORIGINAL or ADAPTIVE_WEIGHTS_DEFINITION_FLIPPED modes. Thank you for finding these.

The fix is currently in https://github.com/how4rd/meshflow/tree/translational-element-fix. I did a test run with ADAPTIVE_WEIGHTS_DEFINITION_ORIGINAL and the results are noticeably smoother! Once I have recreated all the demo videos using the fixed code, I'll merge it into the main branch. It might take a couple days.

how4rd commented 2 years ago

I've merged the fix into the main branch so will close this issue now. Thanks again for letting me know about the bug!