search

moberweger / deep-prior-pp

Improving Fast And Accurate 3D Hand Pose Estimation
GNU General Public License v3.0
144 stars 40 forks source link

Accuracy of Realtime Pipeline #19

Closed anastasia-tkach closed 6 years ago

anastasia-tkach commented 6 years ago

Hi Markus,

Thanks a lot for sharing you code and for explaining in the paper how small changes in the pipeline can significantly improve performance!

I have tested on Linux test_realtimepipeline.py on NUY dataset, using pretrained networks net_NYU_COM_AUGMENT and NYU_network_prior that you have provided. The system compiles and runs without any issues, however the accuracy is a bit lower than in should be, the example results are below: image

  1. Could you please tell, if this performance expected since test_realtimepipeline.py is not using ground truth bounding box? If not, do not maybe have some suggestions of what could have gone wrong and what should I check?

  2. I have run main_nuy_com_refine.py with net_NYU_COM_AUGMENT and got mean error around 5 mm on the first subject and 11 mm on both subjects, is this the expected performance (just confirming, the results look great)?

  3. I attempted to test main_nuy_posereg_embedding.py, but NYU_EMB_t0nF8mp421fD553h1024_PCA30 network is not in provided in pretrained networks. If possible, could you please share it? (I an not attempting to train it, since there seems to be something wrong in my installation of the system.)

Best regards, Anastasia

moberweger commented 6 years ago

Hi Anastasia, thanks for using our code. 1) There are two points: Firstly, did you try using the refinement for the realtime pipeline? I think it is not activated by default. You can do that by pressing 't'. Secondly, there could be several frames, where the assumption that the hand is closest object to the camera is violated due to noise, which then corrupts the results. This is not the case for the training/testing since we do offline preprocessing beforehand. 2) Yes, looks good, should be around 8mm for both subjects on average. 3) The file is NYU_network_prior.pkl and it is provided along with the other files. You don't have to load it in main_nyu_posereg_embedding.py#L145 , but loading the prior network in main_nyu_posereg_embedding.py#L158 is sufficient.

anastasia-tkach commented 6 years ago

Hi Markus,

Thank you so much! I have set tracking to true by pressing 't' and the accuracy became as high as in the paper.

Thanks again for sharing the code and for a prompt reply!

Best regards, Anastasia