ValueError: Tokenizer class `NllbTokenizer` does not exist or is not currently imported when using NLLB (On Paperspace)

[svngoku]

System Info

Hello,

When i use the code below

...
 tokenizer = AutoTokenizer.from_pretrained("facebook/nllb-200-distilled-600M")
...

Models : nllb-distilled-600M

I got an error in my notebook instance (on paperspace) and I thought the problem was with the version of huggingface (4.26.0.dev0) even if I was on the right one it still doesn't work.

🤗

Who can help?

No response

Information

• [X] The official example scripts
• [ ] My own modified scripts

Tasks

• [ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [ ] My own task or dataset (give details below)

Reproduction

...
def load_models():
    # build model and tokenizer
    model_name_dict = {'nllb-distilled-600M': 'facebook/nllb-200-distilled-600M',
                  #'nllb-1.3B': 'facebook/nllb-200-1.3B',
                  #'nllb-distilled-1.3B': 'facebook/nllb-200-distilled-1.3B',
                  #'nllb-3.3B': 'facebook/nllb-200-3.3B',
                  }

    model_dict = {}

    for call_name, real_name in model_name_dict.items():
        print('\tLoading model: %s' % call_name)
        model = AutoModelForSeq2SeqLM.from_pretrained(real_name)
        tokenizer = AutoTokenizer.from_pretrained(real_name)
        model_dict[call_name+'_model'] = model
        model_dict[call_name+'_tokenizer'] = tokenizer

    return model_dict
...

Expected behavior

See my model working as except on the Gradio space

[svngoku]

Bug resolved ! I relaunch my instance many times and run this command !pip3 install git+https://github.com/huggingface/transformers.git

[diarray-hub]

I have the same problem. help!!!