antoinefalisse
commented
1 year ago @suhlrich, this looks good to me. One comment tough. For offline re-processing, we rely on mainSettings.yaml, which is posted after running inverse kinematics. If it crashes before, then even if the pose pickles exist, offline reprocessing will not work. We could post this file earlier to make it work. Only problem is the offset, which is computed from the marker data. We could skip it, but it is quite useful if people want to re-align things. Open to thoughts. One suggestion is to create the yaml at the beginning and update it at the end with the offset. If it crashes then then post yaml in the exception (similar to what you did for the pose pickle files), which will contain all the information we need for offline re-processing. Maybe not the most elegant, but should work.
suhlrich
1) use box-based keypoint tracker with hrnet, (after both box and pose are run) 2) nan low-confidence markers in largest bbox identifier algo (otherwise, spurious keypoints -> large bbox). Seems like openpose may be using .5 confidence threshold during detection (the flag says it is just for visualization, but in several cases i did not see confidence <.5 despite occlusion), which is why this prob didn't show up when we were developing with openpose. 3) solve glob.glob problem with visualizer videos if multiple pose models run locally 4) save pose pickles even when something else fails in main. allows hrnet offline processing in presence of bug.
This makes hrnet more robust to other ppl in frame and errors with bounding box continuity. These are pretty safe changes, as we are just using things we've used with openpose that work well.
I tested on some of the edge cases in parker's data and it works. once we merge to main, I'll move a bunch of his other data through the pipeline and check (I currently can't do that b/c pickles are not online for many of the trials)