关于finetune训练时提示no attribute 'MossTokenizer'、'MossConfig'、'MossForCausalLM'

[Tian14267]

大神们好。最近在finetune训练的时候，经常会提示下面的这些错误：

报错1：

Traceback (most recent call last):
  File "finetune_moss.py", line 305, in <module>
    train(args)           
  File "finetune_moss.py", line 177, in train
    tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path, trust_remote_code=True)
  File "/root/anaconda3/envs/moss/lib/python3.8/site-packages/transformers/models/auto/tokenization_auto.py", line 626, in from_pretrained
    tokenizer_class = get_class_from_dynamic_module(
  File "/root/anaconda3/envs/moss/lib/python3.8/site-packages/transformers/dynamic_module_utils.py", line 374, in get_class_from_dynamic_module
    return get_class_in_module(class_name, final_module.replace(".py", ""))
  File "/root/anaconda3/envs/moss/lib/python3.8/site-packages/transformers/dynamic_module_utils.py", line 148, in get_class_in_module
    return getattr(module, class_name)
AttributeError: module 'transformers_modules.local.tokenization_moss' has no attribute 'MossTokenizer'

报错2：

Traceback (most recent call last):
  File "finetune_moss.py", line 305, in <module>
    train(args)           
  File "finetune_moss.py", line 180, in train
    model = AutoModelForCausalLM.from_pretrained(args.model_name_or_path, trust_remote_code=True, use_cache=False)
  File "/root/anaconda3/envs/moss/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 434, in from_pretrained
    config, kwargs = AutoConfig.from_pretrained(
  File "/root/anaconda3/envs/moss/lib/python3.8/site-packages/transformers/models/auto/configuration_auto.py", line 824, in from_pretrained
    config_class = get_class_from_dynamic_module(
  File "/root/anaconda3/envs/moss/lib/python3.8/site-packages/transformers/dynamic_module_utils.py", line 374, in get_class_from_dynamic_module
    return get_class_in_module(class_name, final_module.replace(".py", ""))
  File "/root/anaconda3/envs/moss/lib/python3.8/site-packages/transformers/dynamic_module_utils.py", line 148, in get_class_in_module
    return getattr(module, class_name)
AttributeError: module 'transformers_modules.local.configuration_moss' has no attribute 'MossConfig'

报错3：

Traceback (most recent call last):
  File "finetune_moss.py", line 305, in <module>
    train(args)           
  File "finetune_moss.py", line 180, in train
    model = AutoModelForCausalLM.from_pretrained(args.model_name_or_path, trust_remote_code=True, use_cache=False)
  File "/root/anaconda3/envs/moss/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 455, in from_pretrained
    model_class = get_class_from_dynamic_module(
  File "/root/anaconda3/envs/moss/lib/python3.8/site-packages/transformers/dynamic_module_utils.py", line 374, in get_class_from_dynamic_module
    return get_class_in_module(class_name, final_module.replace(".py", ""))
  File "/root/anaconda3/envs/moss/lib/python3.8/site-packages/transformers/dynamic_module_utils.py", line 148, in get_class_in_module
    return getattr(module, class_name)
AttributeError: module 'transformers_modules.local.modeling_moss' has no attribute 'MossForCausalLM'

这三个错误，每次都会随机有一个出现。请问是啥情况啊

[xuliang10-tal]

我遇到了 ValueError: Tokenizer class MossTokenizer does not exist or is not currently imported.

[631068264]

+1

num_machines=1
num_processes=$((num_machines * 4))  当这个值=我的显卡数时候就报错，小于不会

[Yiwen-Yang-666]

谁解决了？请教