Fine attribute predictor landmark pooling gives invalid input size

hinneslung commented 4 years ago

I am using the recently added more accurate attribute predictor with landmark pooling. I placed the downloaded model in checkpoint/landmark/resnet50.pth, and a backbone resnet50 model in checkpoint/resnet50.pth, and run the below command:

python demo/test_cate_attr_predictor.py \
    --input demo/imgs/attr_pred_demo2.jpg  \
    --checkpoint checkpoint/landmark/resnet50.pth \
    --config ./configs/category_attribute_predict/roi_predictor_resnet.py

And got the error:

pretrained model checkpoint/resnet50.pth
The model and loaded state dict do not match exactly

unexpected key in source state_dict: fc.weight, fc.bias

model loaded from checkpoint/landmark/resnet50.pth
Traceback (most recent call last):
  File "demo/test_cate_attr_predictor.py", line 65, in <module>
    main()
  File "demo/test_cate_attr_predictor.py", line 55, in main
    attr_prob, cate_prob = model(img_tensor, attr=None,
  File "/home/user/.pyenv/versions/mmfashion-3.8.3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/user/.pyenv/versions/mmfashion-3.8.3/lib/python3.8/site-packages/mmfashion-0.4.0-py3.8.egg/mmfashion/models/predictor/base.py", line 41, in forward
    return self.forward_test(img, landmark)
  File "/home/user/.pyenv/versions/mmfashion-3.8.3/lib/python3.8/site-packages/mmfashion-0.4.0-py3.8.egg/mmfashion/models/predictor/base.py", line 29, in forward_test
    return self.simple_test(img[0], landmark[0])
  File "/home/user/.pyenv/versions/mmfashion-3.8.3/lib/python3.8/site-packages/mmfashion-0.4.0-py3.8.egg/mmfashion/models/predictor/roi_attr_cate_predictor.py", line 56, in simple_test
    attr_pred, cate_pred = self.aug_test(x, landmark)
  File "/home/user/.pyenv/versions/mmfashion-3.8.3/lib/python3.8/site-packages/mmfashion-0.4.0-py3.8.egg/mmfashion/models/predictor/roi_attr_cate_predictor.py", line 64, in aug_test
    local_x = self.roi_pool(x, landmark)
  File "/home/user/.pyenv/versions/mmfashion-3.8.3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/user/.pyenv/versions/mmfashion-3.8.3/lib/python3.8/site-packages/mmfashion-0.4.0-py3.8.egg/mmfashion/models/roi_pool/roi_pooling.py", line 51, in forward
    landmarks = landmarks.view(batch_size, self.num_lms, 2)
RuntimeError: shape '[1, 8, 2]' is invalid for input of size 1

Environment: Python 3.8.3 Package Version
addict 2.2.1
cycler 0.10.0
decorator 4.4.2
flake8 3.8.3
future 0.18.2
imageio 2.9.0
isort 5.3.0
joblib 0.16.0
kiwisolver 1.2.0
matplotlib 3.3.0
mccabe 0.6.1
mmcv 1.0.5
mmfashion 0.4.0
networkx 2.4
numpy 1.19.1
opencv-python 4.2.0.34 Pillow 7.2.0
pip 19.2.3
pycodestyle 2.6.0
pyflakes 2.2.0
pyparsing 2.4.7
python-dateutil 2.8.1
PyWavelets 1.1.1
PyYAML 5.3.1
scikit-image 0.17.2
scikit-learn 0.23.2
scipy 1.5.2
setuptools 41.2.0
six 1.15.0
threadpoolctl 2.1.0
tifffile 2020.7.24 torch 1.5.0
torchvision 0.6.0
yapf 0.30.0

veralauee commented 4 years ago

The pretrained checkpoint --checkpoint checkpoint/landmark/resnet50.pth you used is wrong. This is the model to do landmark prediction, not the model to do category and attribute prediction. Please download the right one from the Model Zoo.

hinneslung commented 4 years ago

Would you mind reviewing the links in model zoo md? I am downloading https://drive.google.com/file/d/1zsgxJAkdumpw4uDkapb1Ulq-aG1Hwz45/view?usp=sharing shown under https://github.com/open-mmlab/mmfashion/blob/master/docs/MODEL_ZOO.md#category-and-attribute-predictionfine

Lytua commented 4 years ago

I got same problem.....

zyue1105 commented 3 years ago

you can change the landmark size to [1, 8, 2] to get the code run https://github.com/open-mmlab/mmfashion/blob/master/demo/test_cate_attr_predictor.py#L44, but that won't give you a good result since the landmark is missing.

my understanding is the demo code needs to first predict landmark https://github.com/open-mmlab/mmfashion/issues/120

JihyeJung-Work commented 3 years ago

The pretrained checkpoint --checkpoint checkpoint/landmark/resnet50.pth you used is wrong. This is the model to do landmark prediction, not the model to do category and attribute prediction. Please download the right one from the Model Zoo.

I have downloaded the right checkpoint file, but occurs the same error.

open-mmlab / mmfashion

Fine attribute predictor landmark pooling gives invalid input size #87