Problem using inference with trained model

carlosrokk3r commented 3 years ago

Hi, @plkmo, first thank you for this repo, it is really nice.

I've trained a model using your script, this way:

python main_task.py \
    --model_no 1 \
    --model_size albert-large-v2 \
    --batch_size 16 \
    --lr 0.00004 \
    --train 1 \
    --infer 0

This generates a model in the data folder, called task_test_model_best_1.pth.tar.

Now I want to infer using this model that I previously trained, I tried doing this the following way:

!python main_task.py \
    --model_no 1 \
    --model_size "/path/to/model/task_test_model_best_1.pth.tar" \
    --train 0 \
    --infer 1

But when doing this, I'm getting the following error:

07/09/2020 10:28:10 PM [INFO]: Loading tokenizer and model...
07/09/2020 10:28:10 PM [INFO]: PyTorch version 1.5.1+cu101 available.
2020-07-09 22:28:10.268412: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
07/09/2020 10:28:11 PM [INFO]: TensorFlow version 2.2.0 available.
Traceback (most recent call last):
  File "main_task.py", line 53, in <module>
    inferer = infer_from_trained(args, detect_entities=True)
  File "/content/BERT-Relation-Extraction/src/tasks/infer.py", line 68, in __init__
    task='classification', n_classes_=self.args.num_classes)
  File "/content/BERT-Relation-Extraction/src/model/ALBERT/modeling_utils.py", line 389, in from_pretrained
    **kwargs,
  File "/content/BERT-Relation-Extraction/src/model/ALBERT/configuration_utils.py", line 176, in from_pretrained
    config_dict, kwargs = cls.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/content/BERT-Relation-Extraction/src/model/ALBERT/configuration_utils.py", line 226, in get_config_dict
    config_dict = cls._dict_from_json_file(resolved_config_file)
  File "/content/BERT-Relation-Extraction/src/model/ALBERT/configuration_utils.py", line 315, in _dict_from_json_file
    text = reader.read()
  File "/usr/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

Am I doing this wrong? Could you please correct me if I'm using this script the wrong way?

plkmo commented 3 years ago

model size for inference should be the same model_size albert-large-v2, the script automatically detects the saved model path so you dont have to pass it as parameter

h3553493 commented 2 years ago

model size for inference should be the same model_size albert-large-v2, the script automatically detects the saved model path so you dont have to pass it as parameter

May I ask why will the model detect the saved model path automatically? I don't see any code that specifies the model path, which makes me confused that how will the infer function load our fine-tuning model?

carlosrokk3r commented 2 years ago

@h3553493 If I'm not mistaken, it loads the fine-tuned model in this line.

The load_state function automatically searches for the best model/last checkpoint and loads it into the NN. This is called from the infer_from_trained class, so the model you get has either the last checkpoint (by default) or the best model (if you change the load_state call to have load_best=True) loaded.

plkmo / BERT-Relation-Extraction

Problem using inference with trained model #21