run_mlm_wwm.py if set(load_result.missing_keys) == set(self.model._keys_to_ignore_on_save): TypeError: 'NoneType' object is not iterable

muyuuuu commented 2 years ago

Environment info

transformers version: 4.7.0
Platform: linux
Python version: 3.9.7
PyTorch version (GPU?): 1.10.2 (cu111)
Tensorflow version (GPU?): no
Using GPU in script?: yes
Using distributed or parallel set-up in script?: no

Who can help

Models:

roberta@LysandreJik

Library:

Trainer: @sgugger

Model hub:

https://huggingface.co/hfl/chinese-roberta-wwm-ext

Examples:

research_projects/mlm_wwm/run_mlm_wwm.py: @julien-c

Information

Model I am using roberta:

The problem arises when using:

[x] the official example scripts: (give details below) https://github.com/huggingface/transformers/tree/main/examples/research_projects/mlm_wwm
[ ] my own modified scripts: (give details below)

The tasks I am working on is:

[x] an official GLUE/SQUaD task: (give the name) chinse-demo.txt as description in here :https://github.com/huggingface/transformers/blob/main/examples/research_projects/mlm_wwm/run_chinese_ref.py#L137
[ ] my own task or dataset: (give details below)

this script has been right:

export TRAIN_FILE=/path/to/train/file
export LTP_RESOURCE=/path/to/ltp/tokenizer
export BERT_RESOURCE=/path/to/bert/tokenizer
export SAVE_PATH=/path/to/data/ref.txt

python run_chinese_ref.py \
    --file_name=$TRAIN_FILE \
    --ltp=$LTP_RESOURCE \
    --bert=$BERT_RESOURCE \
    --save_path=$SAVE_PATH

but got error:

export TRAIN_FILE=/path/to/train/file
export VALIDATION_FILE=/path/to/validation/file
export TRAIN_REF_FILE=/path/to/train/chinese_ref/file
export VALIDATION_REF_FILE=/path/to/validation/chinese_ref/file
export OUTPUT_DIR=/tmp/test-mlm-wwm

python run_mlm_wwm.py \
    --model_name_or_path roberta-base \
    --train_file $TRAIN_FILE \
    --validation_file $VALIDATION_FILE \
    --train_ref_file $TRAIN_REF_FILE \
    --validation_ref_file $VALIDATION_REF_FILE \
    --do_train \
    --do_eval \
    --output_dir $OUTPUT_DIR

error info is :

[INFO|trainer.py:1047] 2022-03-24 17:08:46,271 >> Loading model from chinese-roberta-wwm-ext).
Traceback (most recent call last):
  File "/home/20031211375/pretrain/run_language_modeling.py", line 364, in <module>
    main()
  File "/home/20031211375/pretrain/run_language_modeling.py", line 328, in main
    trainer.train(model_path=model_path)
  File "/home/20031211375/.conda/envs/search/lib/python3.9/site-packages/transformers/trainer.py", line 1066, in train
    self._load_state_dict_in_model(state_dict)
  File "/home/20031211375/.conda/envs/search/lib/python3.9/site-packages/transformers/trainer.py", line 1387, in _load_state_dict_in_model
    if set(load_result.missing_keys) == set(self.model._keys_to_ignore_on_save):
TypeError: 'NoneType' object is not iterable

Expected behavior

how to fix it up?

sgugger commented 2 years ago

I don't think this script works for any other model than BERT as it relies on assumptions that the subwords token have the prefix ##.

GWPunk commented 2 years ago

transformers=4.5.0

github-actions[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

huggingface / transformers