DurhamARC / raga-pose-estimation

MIT License
4 stars 1 forks source link

Investigate removal of jitter on video #9

Closed alisonrclarke closed 4 years ago

alisonrclarke commented 4 years ago

https://stackoverflow.com/questions/52450681/how-can-i-use-smoothing-techniques-to-remove-jitter-in-pose-estimation

MarionBWeinzierl commented 4 years ago

The functionality merged in in #24 could be extended to take more previous dataframes into account for this.

MarionBWeinzierl commented 4 years ago

Alternatively, the result from #10 could be used in the method described in the stackoverflow post.

alisonrclarke commented 4 years ago

I think smoothing might give better results than relying on confidence alone, though it might be that by combining both options we can get even better results.

I'm not sure we need the CSVs to do the smoothing, as we already have the results in a series of data frames. I think the important thing is this line in smooth_pose_data.py:

for i in range(30): df[str(i)] = signal.savgol_filter(df[str(i)], window_length, polyorder)

But it might be that the combining of dataframes that I'm currently doing in CSVWriter for #10 could be done in a separate class at an earlier point.

MarionBWeinzierl commented 4 years ago

I have implemented a smoother using the proposed filter. I might have made a mistake, but so far I cannot get any significant improvement playing with the parameters. The differences are in the decimal places.

MarionBWeinzierl commented 4 years ago

After a lot of frustrated parameter tweaking that did not seem to have an effect I realised that I did not actually write the smoothed frames back into the output array....

With the effect still in the decimal places (I output the correct one to the terminal in my tests....), the effect of smoothing is now visible in the videos. It can be steered by the parameters, but if I increase the window size too much, although it looks very smooth, the movement of the points is then delayed compared to the video (which makes sense). Therefore, I have to make sure to set the parameters to only affect the jitter and not the movement.