SikandAlex commented 4 years ago

Created a new issue because #88 is poorly named

@ashisharora010 @veralauee

I have tried for a week straight to get this repository working and I'm still not having much luck.

Can't run coarse attribute prediction at all

Although the author's claimed 99% accuracy for Top-5 coarse attribute prediction I could not get that demo running at all. Other users have complained of bad predictions and the authors of this code instructed us to use the new Anno_fine data so I've abandoned coarse prediction for now.

See #99

Initial Setup for Fine Attribute Prediction

Download the new Anno_fine folder to use with model https://drive.google.com/drive/folders/19J-FY5NY7s91SiHpQQBo2ad3xjIB42iN

Fine Attributes VGG-16 Global Pooling (poor performance)

1) Download the VGG16 model from PyTorch

wget https://download.pytorch.org/models/vgg16-397923af.pth -O checkpoint/vgg16.pth

2) Download the pre-trained model from Category and Attribute Prediction (Fine) VGG16 Global Pooling from here

https://drive.google.com/file/d/10SZ3Lw4U0F9OKAuHWc-tBbvLS6yfE_x8/view?usp=sharing

wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id=10SZ3Lw4U0F9OKAuHWc-tBbvLS6yfE_x8' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=10SZ3Lw4U0F9OKAuHWc-tBbvLS6yfE_x8" -O finevgg16global.pth && rm -rf /tmp/cookies.txt

3) Prepare a test image

floraltest

4) Get Predictions

python3 test_cate_attr_predictor.py --input floraltest.jpg --checkpoint ./finevgg16global.pth --config ../configs/category_attribute_predict/global_predictor_vgg.py

5) Output

pretrained model checkpoint/vgg16.pth
The model and loaded state dict do not match exactly

unexpected key in source state_dict: classifier.0.weight, classifier.0.bias, classifier.3.weight, classifier.3.bias, classifier.6.weight, classifier.6.bias

model loaded from ./finevgg16global.pth
[ Top3 Attribute Prediction ]
conventional
sleeveless
cotton
[ Top5 Attribute Prediction ]
conventional
sleeveless
cotton
no_neckline
mini_length
[ Top10 Attribute Prediction ]
conventional
sleeveless
cotton
no_neckline
mini_length
floral
maxi_length
crew_neckline
chiffon
solid
[ Top1 Category Prediction ]
Dress
[ Top3 Category Prediction ]
Dress
Jumpsuit
Romper
[ Top5 Category Prediction ]
Dress
Jumpsuit
Romper
Skirt
Blouse

6) Try another test image

jeans

7) Output

pretrained model checkpoint/vgg16.pth
The model and loaded state dict do not match exactly

unexpected key in source state_dict: classifier.0.weight, classifier.0.bias, classifier.3.weight, classifier.3.bias, classifier.6.weight, classifier.6.bias

model loaded from ./finevgg16global.pth
[ Top3 Attribute Prediction ]
no_dress
solid
no_neckline
[ Top5 Attribute Prediction ]
no_dress
solid
no_neckline
conventional
sleeveless
[ Top10 Attribute Prediction ]
no_dress
solid
no_neckline
conventional
sleeveless
cotton
denim
long_sleeve
tight
loose
[ Top1 Category Prediction ]
Jeans
[ Top3 Category Prediction ]
Jeans
Joggers
Shorts
[ Top5 Category Prediction ]
Jeans
Joggers
Shorts
Leggings
Sweatpants

Fine Attributes VGG16 ROI Pooling (does not work at all)

1) Download the VGG16 model from PyTorch

wget https://download.pytorch.org/models/vgg16-397923af.pth -O checkpoint/vgg16.pth

2) Download the pre-trained model from Category and Attribute Prediction (Fine) VGG16 Landmark Pooling from here

https://drive.google.com/file/d/17XlihpZS9iY__i7rPxqlzpenHHRSbLGa/view

wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id=17XlihpZS9iY__i7rPxqlzpenHHRSbLGa' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=17XlihpZS9iY__i7rPxqlzpenHHRSbLGa" -O finevgg16landmark.pth && rm -rf /tmp/cookies.txt

3) Prepare a test image

floraltest

4) Get Predictions

python3 test_cate_attr_predictor.py --input jeans.jpg --checkpoint ./finevgg16landmark.pth --config ../configs/category_attribute_predict/roi_predictor_vgg.py

5) Error / Output

pretrained model checkpoint/vgg16.pth
The model and loaded state dict do not match exactly

unexpected key in source state_dict: classifier.0.weight, classifier.0.bias, classifier.3.weight, classifier.3.bias, classifier.6.weight, classifier.6.bias

model loaded from ./finevgg16landmark.pth
Traceback (most recent call last):
  File "test_cate_attr_predictor.py", line 65, in <module>
    main()
  File "test_cate_attr_predictor.py", line 56, in main
    landmark=landmark_tensor, return_loss=False)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/mmfashion-0.4.0-py3.6.egg/mmfashion/models/predictor/base.py", line 41, in forward
    return self.forward_test(img, landmark)
  File "/usr/local/lib/python3.6/dist-packages/mmfashion-0.4.0-py3.6.egg/mmfashion/models/predictor/base.py", line 29, in forward_test
    return self.simple_test(img[0], landmark[0])
  File "/usr/local/lib/python3.6/dist-packages/mmfashion-0.4.0-py3.6.egg/mmfashion/models/predictor/roi_attr_cate_predictor.py", line 56, in simple_test
    attr_pred, cate_pred = self.aug_test(x, landmark)
  File "/usr/local/lib/python3.6/dist-packages/mmfashion-0.4.0-py3.6.egg/mmfashion/models/predictor/roi_attr_cate_predictor.py", line 64, in aug_test
    local_x = self.roi_pool(x, landmark)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/mmfashion-0.4.0-py3.6.egg/mmfashion/models/roi_pool/roi_pooling.py", line 51, in forward
    landmarks = landmarks.view(batch_size, self.num_lms, 2)
RuntimeError: shape '[1, 8, 2]' is invalid for input of size 1

Fine Attributes ResNet50 Global Pooling (does not work at all)

1) Download the ResNet50 model from PyTorch

wget https://download.pytorch.org/models/resnet50-19c8e357.pth -O checkpoint/resnet50.pth

2) Download the pre-trained model from Category and Attribute Prediction (Fine) ResNet50 Global Pooling from here

https://drive.google.com/file/d/1zsgxJAkdumpw4uDkapb1Ulq-aG1Hwz45/view

wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id=1zsgxJAkdumpw4uDkapb1Ulq-aG1Hwz45' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=1zsgxJAkdumpw4uDkapb1Ulq-aG1Hwz45" -O fineresnet50global.pth && rm -rf /tmp/cookies.txt

3) Prepare a test image

floraltest

4) Get Predictions

python3 test_cate_attr_predictor.py --input jeans.jpg --checkpoint ./fineresnet50global.pth --config ../configs/category_attribute_predict/global_predictor_resnet.py

5) Error / Output

pretrained model checkpoint/resnet50.pth
The model and loaded state dict do not match exactly

unexpected key in source state_dict: fc.weight, fc.bias

The model and loaded state dict do not match exactly

unexpected key in source state_dict: roi_pool.linear.0.weight, roi_pool.linear.0.bias, concat.fc_fusion.weight, concat.fc_fusion.bias

model loaded from ./fineresnet50global.pth

[ Top3 Attribute Prediction ]
striped
pleated
embroidered
[ Top5 Attribute Prediction ]
striped
pleated
embroidered
chiffon
solid
[ Top10 Attribute Prediction ]
striped
pleated
embroidered
chiffon
solid
square_neckline
denim
knit
no_dress
crew_neckline
[ Top1 Category Prediction ]
Cape
[ Top3 Category Prediction ]
Cape
Chinos
Tee
[ Top5 Category Prediction ]
Cape
Chinos
Tee
Shorts
Poncho

Fine Attributes ResNet50 Landmark Pooling (does not work at all)

1) Download the ResNet50 model from PyTorch

wget https://download.pytorch.org/models/resnet50-19c8e357.pth -O checkpoint/resnet50.pth

2) Download the pre-trained model from Category and Attribute Prediction (Fine) ResNet50 Landmark Pooling from here

https://drive.google.com/file/d/1zsgxJAkdumpw4uDkapb1Ulq-aG1Hwz45/view

Note

The ResNet Landmark Pooling / ResNet Global Pooling model links refer to the same file on https://github.com/open-mmlab/mmfashion/blob/master/docs/MODEL_ZOO.md (Not sure if this is an error by authors or not)

wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id=1zsgxJAkdumpw4uDkapb1Ulq-aG1Hwz45' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=1zsgxJAkdumpw4uDkapb1Ulq-aG1Hwz45" -O fineresnet50landmark.pth && rm -rf /tmp/cookies.txt

3) Prepare a test image

floraltest

4) Get Predictions

python3 test_cate_attr_predictor.py --input jeans.jpg --checkpoint ./fineresnet50landmark.pth --config ../configs/category_attribute_predict/roi_predictor_resnet.py

5) Error / Output

pretrained model checkpoint/resnet50.pth
The model and loaded state dict do not match exactly

unexpected key in source state_dict: fc.weight, fc.bias

The model and loaded state dict do not match exactly

unexpected key in source state_dict: roi_pool.linear.0.weight, roi_pool.linear.0.bias, concat.fc_fusion.weight, concat.fc_fusion.bias

model loaded from ./fineresnetlandmark.pth
[ Top3 Attribute Prediction ]
striped
denim
no_dress
[ Top5 Attribute Prediction ]
striped
denim
no_dress
embroidered
solid
[ Top10 Attribute Prediction ]
striped
denim
no_dress
embroidered
solid
square_neckline
chiffon
v_neckline
pleated
knit
[ Top1 Category Prediction ]
Cape
[ Top3 Category Prediction ]
Cape
Tee
Jeggings
[ Top5 Category Prediction ]
Cape
Tee
Jeggings
Chinos
Poncho

steve-1820 commented 4 years ago

I am having this exact issue - the results are all weird.

I followed the getting started.

steve-1820 commented 4 years ago

I get similar results to the above for the resnet50 global pooling and an error for the vgg landmark pooling:

RuntimeError: shape '[1, 8, 2]' is invalid for input of size 1

SikandAlex commented 3 years ago

I'll try to replicate the results on the test dataset but it's really frustrating trying to get a lot of these research papers to work in code. Half the time its poorly supported by the original authors and half the time its simply not reproducible. I really appreciate the authors work on this project and it is impressive for sure but as an MS Candidate in Artificial Intelligence and fellow scholar I really wish this library could be improved and I could understand some of the errors I'm getting or why I'm getting pretty bad predictions for the attributes.

2020-wx commented 3 years ago

Having been benefited from the authors efforts, very much appreciated in the first place. While frankly speaking, the quality of the code is kind of undermining the value of the work. Got similar experiences to @SikandAlex, running through all the fine classification recipes, and only vgg16+global worked out and gave stable results - other recipes gave unstable results between different runs and the results just look wrong. Do wish this code repo can be improved to a next level.

Karenou commented 2 years ago

I get similar results to the above for the resnet50 global pooling and an error for the vgg landmark pooling:

RuntimeError: shape '[1, 8, 2]' is invalid for input of size 1

I also got the same error, may I know did you solve it in the end? Many thanks!

Karenou commented 2 years ago

Hi Xia,

I solved the problem. You need to change the landmark tensor to shape of 16 instead of 8 in test_attr_predictor.py when using the roi model.

landmark_tensor = torch.zeros(16).view(1,-1)

Best, Karen

Baiqiang XIA @.***>于2022年2月9日周三下午6:24写道：

I get similar results to the above for the resnet50 global pooling and an error for the vgg landmark pooling: RuntimeError: shape '[1, 8, 2]' is invalid for input of size 1

I also got the same error, may I know did you solve it in the end? Many thanks!

Hi, I tried many times but it was not fixable, so had to give up at the end. PS: [2020-wx] was my old account in Github.

— Reply to this email directly, view it on GitHub https://github.com/open-mmlab/mmfashion/issues/100#issuecomment-1033602092, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKTNYE24Z6NUE4TZGUOWE23U2I6GXANCNFSM4RZIWUSQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

open-mmlab / mmfashion

Can't run Fine Attribute Detection properly #100

Can't run coarse attribute prediction at all

Initial Setup for Fine Attribute Prediction

Fine Attributes VGG-16 Global Pooling (poor performance)

1) Download the VGG16 model from PyTorch

2) Download the pre-trained model from Category and Attribute Prediction (Fine) VGG16 Global Pooling from here

3) Prepare a test image

4) Get Predictions

5) Output

6) Try another test image

7) Output

Fine Attributes VGG16 ROI Pooling (does not work at all)

1) Download the VGG16 model from PyTorch

2) Download the pre-trained model from Category and Attribute Prediction (Fine) VGG16 Landmark Pooling from here

3) Prepare a test image

4) Get Predictions

5) Error / Output

Fine Attributes ResNet50 Global Pooling (does not work at all)

1) Download the ResNet50 model from PyTorch

2) Download the pre-trained model from Category and Attribute Prediction (Fine) ResNet50 Global Pooling from here

3) Prepare a test image

4) Get Predictions

5) Error / Output

Fine Attributes ResNet50 Landmark Pooling (does not work at all)

1) Download the ResNet50 model from PyTorch

2) Download the pre-trained model from Category and Attribute Prediction (Fine) ResNet50 Landmark Pooling from here

Note

3) Prepare a test image

4) Get Predictions

5) Error / Output