search

clovaai / deep-text-recognition-benchmark

Text recognition (optical character recognition) with deep learning methods, ICCV 2019
Apache License 2.0
3.75k stars 1.1k forks source link

Unable to load the best accuracy model for prediction #298

Open ashwiniimnair opened 3 years ago

ashwiniimnair commented 3 years ago

Trained the model with custom dataset and when trying to test it with demo.py file, It gives an the below error

Error

['model input parameters 32 100 20 1 512 256 38 25 TPS ResNet BiLSTM Attn', 'loading pretrained model from /content/deep-text-recognition-benchmark/best_accuracy.pth', 'Traceback (most recent call last):', ' File "/content/deep-text-recognition-benchmark/demo.py", line 129, in ', ' demo(opt)', ' File "/content/deep-text-recognition-benchmark/demo.py", line 33, in demo', ' model.load_state_dict(torch.load(opt.saved_model, map_location=device))', ' File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1407, in load_state_dict', ' self.class.name, "\n\t".join(error_msgs)))', 'RuntimeError: Error(s) in loading state_dict for DataParallel:', '\tMissing key(s) in state_dict: "module.Transformation.LocalizationNetwork.conv.0.weight", "module.Transformation.LocalizationNetwork.conv.1.weight", "module.Transformation.LocalizationNetwork.conv.1.bias", "module.Transformation.LocalizationNetwork.conv.1.running_mean", "module.Transformation.LocalizationNetwork.conv.1.running_var", "module.Transformation.LocalizationNetwork.conv.4.weight", "module.Transformation.LocalizationNetwork.conv.5.weight", "module.Transformation.LocalizationNetwork.conv.5.bias", "module.Transformation.LocalizationNetwork.conv.5.running_mean", "module.Transformation.LocalizationNetwork.conv.5.running_var", "module.Transformation.LocalizationNetwork.conv.8.weight", "module.Transformation.LocalizationNetwork.conv.9.weight", "module.Transformation.LocalizationNetwork.conv.9.bias", "module.Transformation.LocalizationNetwork.conv.9.running_mean", "module.Transformation.LocalizationNetwork.conv.9.running_var", "module.Transformation.LocalizationNetwork.conv.12.weight", "module.Transformation.LocalizationNetwork.conv.13.weight", "module.Transformation.LocalizationNetwork.conv.13.bias", "module.Transformation.LocalizationNetwork.conv.13.running_mean", "module.Transformation.LocalizationNetwork.conv.13.running_var", "module.Transformation.LocalizationNetwork.localization_fc1.0.weight", "module.Transformation.LocalizationNetwork.localization_fc1.0.bias", "module.Transformation.LocalizationNetwork.localization_fc2.weight", "module.Transformation.LocalizationNetwork.localization_fc2.bias", "module.Transformation.GridGenerator.inv_delta_C", "module.Transformation.GridGenerator.P_hat", "module.FeatureExtraction.ConvNet.conv0_1.weight", "module.FeatureExtraction.ConvNet.bn0_1.weight", "module.FeatureExtraction.ConvNet.bn0_1.bias", "module.FeatureExtraction.ConvNet.bn0_1.running_mean", "module.FeatureExtraction.ConvNet.bn0_1.running_var", "module.FeatureExtraction.ConvNet.conv0_2.weight", "module.FeatureExtraction.ConvNet.bn0_2.weight", "module.FeatureExtraction.ConvNet.bn0_2.bias", "module.FeatureExtraction.ConvNet.bn0_2.running_mean", "module.FeatureExtraction.ConvNet.bn0_2.running_var", "module.FeatureExtraction.ConvNet.layer1.0.conv1.weight", "module.FeatureExtraction.ConvNet.layer1.0.bn1.weight", "module.FeatureExtraction.ConvNet.layer1.0.bn1.bias", "module.FeatureExtraction.ConvNet.layer1.0.bn1.running_mean", "module.FeatureExtraction.ConvNet.layer1.0.bn1.running_var", "module.FeatureExtraction.ConvNet.layer1.0.conv2.weight", "module.FeatureExtraction.ConvNet.layer1.0.bn2.weight", "module.FeatureExtraction.ConvNet.layer1.0.bn2.bias", "module.FeatureExtraction.ConvNet.layer1.0.bn2.running_mean", "module.FeatureExtraction.ConvNet.layer1.0.bn2.running_var", "module.FeatureExtraction.ConvNet.layer1.0.downsample.0.weight", "module.FeatureExtraction.ConvNet.layer1.0.downsample.1.weight", "module.FeatureExtraction.ConvNet.layer1.0.downsample.1.bias", "module.FeatureExtraction.ConvNet.layer1.0.downsample.1.running_mean", "module.FeatureExtraction.ConvNet.layer1.0.downsample.1.running_var", "module.FeatureExtraction.ConvNet.conv1.weight", "module.FeatureExtraction.ConvNet.bn1.weight", "module.FeatureExtraction.ConvNet.bn1.bias", "module.FeatureExtraction.ConvNet.bn1.running_mean", "module.FeatureExtraction.ConvNet.bn1.running_var", "module.FeatureExtraction.ConvNet.layer2.0.conv1.weight", "module.FeatureExtraction.ConvNet.layer2.0.bn1.weight", "module.FeatureExtraction.ConvNet.layer2.0.bn1.bias", "module.FeatureExtraction.ConvNet.layer2.0.bn1.running_mean", "module.FeatureExtraction.ConvNet.layer2.0.bn1.running_var", "module.FeatureExtraction.ConvNet.layer2.0.conv2.weight", "module.FeatureExtraction.ConvNet.layer2.0.bn2.weight", "module.FeatureExtraction.ConvNet.layer2.0.bn2.bias", "module.FeatureExtraction.ConvNet.layer2.0.bn2.running_mean", "module.FeatureExtraction.ConvNet.layer2.0.bn2.running_var", "module.FeatureExtraction.ConvNet.layer2.0.downsample.0.weight", "module.FeatureExtraction.ConvNet.layer2.0.downsample.1.weight", "module.FeatureExtraction.ConvNet.layer2.0.downsample.1.bias", "module.FeatureExtraction.ConvNet.layer2.0.downsample.1.running_mean", "module.FeatureExtraction.ConvNet.layer2.0.downsample.1.running_var", "module.FeatureExtraction.ConvNet.layer2.1.conv1.weight", "module.FeatureExtraction.ConvNet.layer2.1.bn1.weight", "module.FeatureExtraction.ConvNet.layer2.1.bn1.bias", "module.FeatureExtraction.ConvNet.layer2.1.bn1.running_mean", "module.FeatureExtraction.ConvNet.layer2.1.bn1.running_var", "module.FeatureExtraction.ConvNet.layer2.1.conv2.weight", "module.FeatureExtraction.ConvNet.layer2.1.bn2.weight", "module.FeatureExtraction.ConvNet.layer2.1.bn2.bias", "module.FeatureExtraction.ConvNet.layer2.1.bn2.running_mean", "module.FeatureExtraction.ConvNet.layer2.1.bn2.running_var", "module.FeatureExtraction.ConvNet.conv2.weight", "module.FeatureExtraction.ConvNet.bn2.weight", "module.FeatureExtraction.ConvNet.bn2.bias", "module.FeatureExtraction.ConvNet.bn2.running_mean", "module.FeatureExtraction.ConvNet.bn2.running_var", "module.FeatureExtraction.ConvNet.layer3.0.conv1.weight", "module.FeatureExtraction.ConvNet.layer3.0.bn1.weight", "module.FeatureExtraction.ConvNet.layer3.0.bn1.bias", "module.FeatureExtraction.ConvNet.layer3.0.bn1.running_mean", "module.FeatureExtraction.ConvNet.layer3.0.bn1.running_var", "module.FeatureExtraction.ConvNet.layer3.0.conv2.weight", "module.FeatureExtraction.ConvNet.layer3.0.bn2.weight", "module.FeatureExtraction.ConvNet.layer3.0.bn2.bias", "module.FeatureExtraction.ConvNet.layer3.0.bn2.running_mean", "module.FeatureExtraction.ConvNet.layer3.0.bn2.running_var", "module.FeatureExtraction.ConvNet.layer3.0.downsample.0.weight", "module.FeatureExtraction.ConvNet.layer3.0.downsample.1.weight", "module.FeatureExtraction.ConvNet.layer3.0.downsample.1.bias", "module.FeatureExtraction.ConvNet.layer3.0.downsample.1.running_mean", "module.FeatureExtraction.ConvNet.layer3.0.downsample.1.running_var", "module.FeatureExtraction.ConvNet.layer3.1.conv1.weight", "module.FeatureExtraction.ConvNet.layer3.1.bn1.weight", "module.FeatureExtraction.ConvNet.layer3.1.bn1.bias", "module.FeatureExtraction.ConvNet.layer3.1.bn1.running_mean", "module.FeatureExtraction.ConvNet.layer3.1.bn1.running_var", "module.FeatureExtraction.ConvNet.layer3.1.conv2.weight", "module.FeatureExtraction.ConvNet.layer3.1.bn2.weight", "module.FeatureExtraction.ConvNet.layer3.1.bn2.bias", "module.FeatureExtraction.ConvNet.layer3.1.bn2.running_mean", "module.FeatureExtraction.ConvNet.layer3.1.bn2.running_var", "module.FeatureExtraction.ConvNet.layer3.2.conv1.weight", "module.FeatureExtraction.ConvNet.layer3.2.bn1.weight", "module.FeatureExtraction.ConvNet.layer3.2.bn1.bias", "module.FeatureExtraction.ConvNet.layer3.2.bn1.running_mean", "module.FeatureExtraction.ConvNet.layer3.2.bn1.running_var", "module.FeatureExtraction.ConvNet.layer3.2.conv2.weight", "module.FeatureExtraction.ConvNet.layer3.2.bn2.weight", "module.FeatureExtraction.ConvNet.layer3.2.bn2.bias", "module.FeatureExtraction.ConvNet.layer3.2.bn2.running_mean", "module.FeatureExtraction.ConvNet.layer3.2.bn2.running_var", "module.FeatureExtraction.ConvNet.layer3.3.conv1.weight", "module.FeatureExtraction.ConvNet.layer3.3.bn1.weight", "module.FeatureExtraction.ConvNet.layer3.3.bn1.bias", "module.FeatureExtraction.ConvNet.layer3.3.bn1.running_mean", "module.FeatureExtraction.ConvNet.layer3.3.bn1.running_var", "module.FeatureExtraction.ConvNet.layer3.3.conv2.weight", "module.FeatureExtraction.ConvNet.layer3.3.bn2.weight", "module.FeatureExtraction.ConvNet.layer3.3.bn2.bias", "module.FeatureExtraction.ConvNet.layer3.3.bn2.running_mean", "module.FeatureExtraction.ConvNet.layer3.3.bn2.running_var", "module.FeatureExtraction.ConvNet.layer3.4.conv1.weight", "module.FeatureExtraction.ConvNet.layer3.4.bn1.weight", "module.FeatureExtraction.ConvNet.layer3.4.bn1.bias", "module.FeatureExtraction.ConvNet.layer3.4.bn1.running_mean", "module.FeatureExtraction.ConvNet.layer3.4.bn1.running_var", "module.FeatureExtraction.ConvNet.layer3.4.conv2.weight", "module.FeatureExtraction.ConvNet.layer3.4.bn2.weight", "module.FeatureExtraction.ConvNet.layer3.4.bn2.bias", "module.FeatureExtraction.ConvNet.layer3.4.bn2.running_mean", "module.FeatureExtraction.ConvNet.layer3.4.bn2.running_var", "module.FeatureExtraction.ConvNet.conv3.weight", "module.FeatureExtraction.ConvNet.bn3.weight", "module.FeatureExtraction.ConvNet.bn3.bias", "module.FeatureExtraction.ConvNet.bn3.running_mean", "module.FeatureExtraction.ConvNet.bn3.running_var", "module.FeatureExtraction.ConvNet.layer4.0.conv1.weight", "module.FeatureExtraction.ConvNet.layer4.0.bn1.weight", "module.FeatureExtraction.ConvNet.layer4.0.bn1.bias", "module.FeatureExtraction.ConvNet.layer4.0.bn1.running_mean", "module.FeatureExtraction.ConvNet.layer4.0.bn1.running_var", "module.FeatureExtraction.ConvNet.layer4.0.conv2.weight", "module.FeatureExtraction.ConvNet.layer4.0.bn2.weight", "module.FeatureExtraction.ConvNet.layer4.0.bn2.bias", "module.FeatureExtraction.ConvNet.layer4.0.bn2.running_mean", "module.FeatureExtraction.ConvNet.layer4.0.bn2.running_var", "module.FeatureExtraction.ConvNet.layer4.1.conv1.weight", "module.FeatureExtraction.ConvNet.layer4.1.bn1.weight", "module.FeatureExtraction.ConvNet.layer4.1.bn1.bias", "module.FeatureExtraction.ConvNet.layer4.1.bn1.running_mean", "module.FeatureExtraction.ConvNet.layer4.1.bn1.running_var", "module.FeatureExtraction.ConvNet.layer4.1.conv2.weight", "module.FeatureExtraction.ConvNet.layer4.1.bn2.weight", "module.FeatureExtraction.ConvNet.layer4.1.bn2.bias", "module.FeatureExtraction.ConvNet.layer4.1.bn2.running_mean", "module.FeatureExtraction.ConvNet.layer4.1.bn2.running_var", "module.FeatureExtraction.ConvNet.layer4.2.conv1.weight", "module.FeatureExtraction.ConvNet.layer4.2.bn1.weight", "module.FeatureExtraction.ConvNet.layer4.2.bn1.bias", "module.FeatureExtraction.ConvNet.layer4.2.bn1.running_mean", "module.FeatureExtraction.ConvNet.layer4.2.bn1.running_var", "module.FeatureExtraction.ConvNet.layer4.2.conv2.weight", "module.FeatureExtraction.ConvNet.layer4.2.bn2.weight", "module.FeatureExtraction.ConvNet.layer4.2.bn2.bias", "module.FeatureExtraction.ConvNet.layer4.2.bn2.running_mean", "module.FeatureExtraction.ConvNet.layer4.2.bn2.running_var", "module.FeatureExtraction.ConvNet.conv4_1.weight", "module.FeatureExtraction.ConvNet.bn4_1.weight", "module.FeatureExtraction.ConvNet.bn4_1.bias", "module.FeatureExtraction.ConvNet.bn4_1.running_mean", "module.FeatureExtraction.ConvNet.bn4_1.running_var", "module.FeatureExtraction.ConvNet.conv4_2.weight", "module.FeatureExtraction.ConvNet.bn4_2.weight", "module.FeatureExtraction.ConvNet.bn4_2.bias", "module.FeatureExtraction.ConvNet.bn4_2.running_mean", "module.FeatureExtraction.ConvNet.bn4_2.running_var", "module.Prediction.attention_cell.i2h.weight", "module.Prediction.attention_cell.h2h.weight", "module.Prediction.attention_cell.h2h.bias", "module.Prediction.attention_cell.score.weight", "module.Prediction.attention_cell.rnn.weight_ih", "module.Prediction.attention_cell.rnn.weight_hh", "module.Prediction.attention_cell.rnn.bias_ih", "module.Prediction.attention_cell.rnn.bias_hh", "module.Prediction.generator.weight", "module.Prediction.generator.bias". ', '\tUnexpected key(s) in state_dict: "module.FeatureExtraction.ConvNet.0.weight", "module.FeatureExtraction.ConvNet.0.bias", "module.FeatureExtraction.ConvNet.3.weight", "module.FeatureExtraction.ConvNet.3.bias", "module.FeatureExtraction.ConvNet.6.weight", "module.FeatureExtraction.ConvNet.6.bias", "module.FeatureExtraction.ConvNet.8.weight", "module.FeatureExtraction.ConvNet.8.bias", "module.FeatureExtraction.ConvNet.11.weight", "module.FeatureExtraction.ConvNet.12.weight", "module.FeatureExtraction.ConvNet.12.bias", "module.FeatureExtraction.ConvNet.12.running_mean", "module.FeatureExtraction.ConvNet.12.running_var", "module.FeatureExtraction.ConvNet.12.num_batches_tracked", "module.FeatureExtraction.ConvNet.14.weight", "module.FeatureExtraction.ConvNet.15.weight", "module.FeatureExtraction.ConvNet.15.bias", "module.FeatureExtraction.ConvNet.15.running_mean", "module.FeatureExtraction.ConvNet.15.running_var", "module.FeatureExtraction.ConvNet.15.num_batches_tracked", "module.FeatureExtraction.ConvNet.18.weight", "module.FeatureExtraction.ConvNet.18.bias", "module.Prediction.weight", "module.Prediction.bias". ']

I am able to solve the loading problem by changing the demo.py file with below code

 print('loading pretrained model from %s' % opt.saved_model)       
 checkpoint = torch.load(opt.saved_model, map_location=device) # Load the model file, pt, pth file can be;

 if torch.cuda.device_count() > 1:
    model = nn.DataParallel(model) 
    model.load_state_dict(checkpoint)

However the predictions are going wrong/throwing garbage value after that. Please help

uranutan commented 3 years ago

I have exactly the same problem!!

ashwiniimnair commented 3 years ago

The problem was solved. We need to train the custom model by keeping argument --sensitive false. ie, parser.add_argument('--sensitive', default=False , action='store_true', help='for sensitive character mode') also change the above line in demo.py as well.

uranutan commented 3 years ago

The problem was solved. We need to train the custom model by keeping argument --sensitive false. ie, parser.add_argument('--sensitive', default=False , action='store_true', help='for sensitive character mode') also change the above line in demo.py as well.

I just want to do inference with the pretrained weight. But I am unable to get any sensible predicted labels at all. I manage to load the models after a while.

uranutan commented 3 years ago

This issue occurs only when I run on new computer with Mac M1 chip. It is hard to tell exactly which function is running into problems. Otherwise it is running perfectly.