Compatibility enhancement and bug fix

jiang-du commented 4 years ago

Compatibility enhanced. The original version could only use package "scipy" with no more than 1.2 version. Otherwise, images could not be read and resized. While, this commit allows the latest version (1.4.1). Also, python 3.8.2 and pytorch 1.5 under Ubuntu 20.04 has been tested to be fine.
Fix a bug. The original version forgets to "import shutil" in train.py. This has been added.

crockwell commented 4 years ago

Thanks jiang-du, this is really helpful. Have you checked to see if loaded and resized images look the same as before? I know there can be small discrepancies between methods, so my only concern is it changing the ability to replicate results. If the images aren't the same, would you mind seeing how similar they are / how similar results you can get?

jiang-du commented 4 years ago

According to the document of the functions, I think the resized image may not be exactly the same as before. This could be because of the different interpolation methods.

In the original version, scipy.misc.imresize() is employed. While, according to the official document, I cannot find which interpolation to use for resizing (‘nearest’, ‘lanczos’, ‘bilinear’, ‘bicubic’ or ‘cubic’).
In the modified version, I use cv2.resize(). By default, it uses INTER_LINEAR (bilinear interpolation).

The results are similar. Using the given 8HG pretrained model, the original validation accuracy was said to be 0.901. While, in the modified code, the validation accuracy is 0.902. The following are my detailed validation results.

all :
Val PCK @, 0.5 , total : 0.902 , count: 44239
Tra PCK @, 0.5 , total : 0.963 , count: 4443
Val PCK @, 0.5 , ankle : 0.808 , count: 4234
Tra PCK @, 0.5 , ankle : 0.903 , count: 392
Val PCK @, 0.5 , knee : 0.846 , count: 4963
Tra PCK @, 0.5 , knee : 0.948 , count: 499
Val PCK @, 0.5 , hip : 0.888 , count: 5777
Tra PCK @, 0.5 , hip : 0.947 , count: 587
Val PCK @, 0.5 , pelvis : 0.926 , count: 2878
Tra PCK @, 0.5 , pelvis : 0.98 , count: 293
Val PCK @, 0.5 , thorax : 0.985 , count: 2932
Tra PCK @, 0.5 , thorax : 0.997 , count: 298
Val PCK @, 0.5 , neck : 0.98 , count: 2932
Tra PCK @, 0.5 , neck : 0.997 , count: 298
Val PCK @, 0.5 , head : 0.965 , count: 2931
Tra PCK @, 0.5 , head : 0.99 , count: 297
Val PCK @, 0.5 , wrist : 0.85 , count: 5837
Tra PCK @, 0.5 , wrist : 0.937 , count: 589
Val PCK @, 0.5 , elbow : 0.903 , count: 5867
Tra PCK @, 0.5 , elbow : 0.97 , count: 592
Val PCK @, 0.5 , shoulder : 0.958 , count: 5888
Tra PCK @, 0.5 , shoulder : 0.993 , count: 598

visible :
Val PCK @, 0.5 , total : 0.944 , count: 33996
Tra PCK @, 0.5 , total : 0.973 , count: 3879
Val PCK @, 0.5 , ankle : 0.892 , count: 3177
Tra PCK @, 0.5 , ankle : 0.938 , count: 340
Val PCK @, 0.5 , knee : 0.907 , count: 3854
Tra PCK @, 0.5 , knee : 0.964 , count: 448
Val PCK @, 0.5 , hip : 0.919 , count: 4206
Tra PCK @, 0.5 , hip : 0.94 , count: 503
Val PCK @, 0.5 , pelvis : 0.956 , count: 1620
Tra PCK @, 0.5 , pelvis : 0.977 , count: 221
Val PCK @, 0.5 , thorax : 0.992 , count: 1691
Tra PCK @, 0.5 , thorax : 1.0 , count: 236
Val PCK @, 0.5 , neck : 0.98 , count: 2932
Tra PCK @, 0.5 , neck : 0.997 , count: 298
Val PCK @, 0.5 , head : 0.965 , count: 2931
Tra PCK @, 0.5 , head : 0.99 , count: 297
Val PCK @, 0.5 , wrist : 0.938 , count: 4395
Tra PCK @, 0.5 , wrist : 0.969 , count: 490
Val PCK @, 0.5 , elbow : 0.952 , count: 4590
Tra PCK @, 0.5 , elbow : 0.977 , count: 511
Val PCK @, 0.5 , shoulder : 0.974 , count: 4600
Tra PCK @, 0.5 , shoulder : 0.994 , count: 535

not visible :
Val PCK @, 0.5 , total : 0.762 , count: 10243
Tra PCK @, 0.5 , total : 0.895 , count: 564
Val PCK @, 0.5 , ankle : 0.558 , count: 1057
Tra PCK @, 0.5 , ankle : 0.673 , count: 52
Val PCK @, 0.5 , knee : 0.634 , count: 1109
Tra PCK @, 0.5 , knee : 0.804 , count: 51
Val PCK @, 0.5 , hip : 0.805 , count: 1571
Tra PCK @, 0.5 , hip : 0.988 , count: 84
Val PCK @, 0.5 , pelvis : 0.887 , count: 1258
Tra PCK @, 0.5 , pelvis : 0.986 , count: 72
Val PCK @, 0.5 , thorax : 0.975 , count: 1241
Tra PCK @, 0.5 , thorax : 0.984 , count: 62
Val PCK @, 0.5 , neck : 0.0 , count: 0
Tra PCK @, 0.5 , neck : 0.0 , count: 0
Val PCK @, 0.5 , head : 0.0 , count: 0
Tra PCK @, 0.5 , head : 0.0 , count: 0
Val PCK @, 0.5 , wrist : 0.583 , count: 1442
Tra PCK @, 0.5 , wrist : 0.778 , count: 99
Val PCK @, 0.5 , elbow : 0.724 , count: 1277
Tra PCK @, 0.5 , elbow : 0.926 , count: 81
Val PCK @, 0.5 , shoulder : 0.898 , count: 1288
Tra PCK @, 0.5 , shoulder : 0.984 , count: 63

jiang-du commented 4 years ago

I have found another detail problem in the modified code, which would cause this error when training on MPII dataset:

TypeError: Invalid keyword argument "mode", perhaps you mean "pilmode"?

The solution is to modify line 93 in /pytorch_stacked_hourglass/data/MPII/ref.py to img = imread(path).

crockwell commented 4 years ago

Looks good, thanks again! Just so I can update the README, can you please report the validation accuracy of the 2HG network with the updates? Then I can merge the pull request.

jiang-du commented 4 years ago

Certainly, I have tested the 2HG model. While in this model, it is unfortunately a little bit lower than the original results. The details of 2HG results are as follows.

all :
Val PCK @, 0.5 , total : 0.883 , count: 44239
Tra PCK @, 0.5 , total : 0.941 , count: 4443
Val PCK @, 0.5 , ankle : 0.779 , count: 4234
Tra PCK @, 0.5 , ankle : 0.857 , count: 392
Val PCK @, 0.5 , knee : 0.812 , count: 4963
Tra PCK @, 0.5 , knee : 0.914 , count: 499
Val PCK @, 0.5 , hip : 0.874 , count: 5777
Tra PCK @, 0.5 , hip : 0.918 , count: 587
Val PCK @, 0.5 , pelvis : 0.91 , count: 2878
Tra PCK @, 0.5 , pelvis : 0.973 , count: 293
Val PCK @, 0.5 , thorax : 0.98 , count: 2932
Tra PCK @, 0.5 , thorax : 0.997 , count: 298
Val PCK @, 0.5 , neck : 0.975 , count: 2932
Tra PCK @, 0.5 , neck : 0.993 , count: 298
Val PCK @, 0.5 , head : 0.957 , count: 2931
Tra PCK @, 0.5 , head : 0.99 , count: 297
Val PCK @, 0.5 , wrist : 0.82 , count: 5837
Tra PCK @, 0.5 , wrist : 0.898 , count: 589
Val PCK @, 0.5 , elbow : 0.878 , count: 5867
Tra PCK @, 0.5 , elbow : 0.946 , count: 592
Val PCK @, 0.5 , shoulder : 0.951 , count: 5888
Tra PCK @, 0.5 , shoulder : 0.987 , count: 598

visible :
Val PCK @, 0.5 , total : 0.931 , count: 33996
Tra PCK @, 0.5 , total : 0.96 , count: 3879
Val PCK @, 0.5 , ankle : 0.87 , count: 3177
Tra PCK @, 0.5 , ankle : 0.909 , count: 340
Val PCK @, 0.5 , knee : 0.882 , count: 3854
Tra PCK @, 0.5 , knee : 0.935 , count: 448
Val PCK @, 0.5 , hip : 0.908 , count: 4206
Tra PCK @, 0.5 , hip : 0.924 , count: 503
Val PCK @, 0.5 , pelvis : 0.945 , count: 1620
Tra PCK @, 0.5 , pelvis : 0.973 , count: 221
Val PCK @, 0.5 , thorax : 0.989 , count: 1691
Tra PCK @, 0.5 , thorax : 1.0 , count: 236
Val PCK @, 0.5 , neck : 0.975 , count: 2932
Tra PCK @, 0.5 , neck : 0.993 , count: 298
Val PCK @, 0.5 , head : 0.957 , count: 2931
Tra PCK @, 0.5 , head : 0.99 , count: 297
Val PCK @, 0.5 , wrist : 0.919 , count: 4395
Tra PCK @, 0.5 , wrist : 0.953 , count: 490
Val PCK @, 0.5 , elbow : 0.937 , count: 4590
Tra PCK @, 0.5 , elbow : 0.963 , count: 511
Val PCK @, 0.5 , shoulder : 0.967 , count: 4600
Tra PCK @, 0.5 , shoulder : 0.991 , count: 535

not visible :
Val PCK @, 0.5 , total : 0.726 , count: 10243
Tra PCK @, 0.5 , total : 0.814 , count: 564
Val PCK @, 0.5 , ankle : 0.504 , count: 1057
Tra PCK @, 0.5 , ankle : 0.519 , count: 52
Val PCK @, 0.5 , knee : 0.571 , count: 1109
Tra PCK @, 0.5 , knee : 0.725 , count: 51
Val PCK @, 0.5 , hip : 0.785 , count: 1571
Tra PCK @, 0.5 , hip : 0.881 , count: 84
Val PCK @, 0.5 , pelvis : 0.865 , count: 1258
Tra PCK @, 0.5 , pelvis : 0.972 , count: 72
Val PCK @, 0.5 , thorax : 0.968 , count: 1241
Tra PCK @, 0.5 , thorax : 0.984 , count: 62
Val PCK @, 0.5 , neck : 0.0 , count: 0
Tra PCK @, 0.5 , neck : 0.0 , count: 0
Val PCK @, 0.5 , head : 0.0 , count: 0
Tra PCK @, 0.5 , head : 0.0 , count: 0
Val PCK @, 0.5 , wrist : 0.519 , count: 1442
Tra PCK @, 0.5 , wrist : 0.626 , count: 99
Val PCK @, 0.5 , elbow : 0.662 , count: 1277
Tra PCK @, 0.5 , elbow : 0.84 , count: 81
Val PCK @, 0.5 , shoulder : 0.894 , count: 1288
Tra PCK @, 0.5 , shoulder : 0.952 , count: 63

princeton-vl / pytorch_stacked_hourglass

Compatibility enhancement and bug fix #14