NLLB Error while translating: "thammegowda-nllb-serve" requires a web server running locally!

[LeitHunt]

Followed this Tutorial Getting Error: "thammegowda-nllb-serve" requires a web server running locally! Trying to Translate Japanese to English

In my CMD:

Microsoft Windows [Version 10.0.22631.4169] (c) Microsoft Corporation. All rights reserved.

C:\Users\xxxx>nllb-serve INFO:root:torch device=cuda INFO:root:Loading model facebook/nllb-200-distilled-600M ... INFO:root:Loading default tokenizer for facebook/nllb-200-distilled-600M ... C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\tokenization_utils_base.py:1601: FutureWarning: clean_up_tokenization_spaces was not set. It will be set to True by default. This behavior will be depracted in transformers v4.45, and will be then set to False by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884 warnings.warn( INFO:root:System Info: ${'transformer': '4.44.2', 'Python Version': '3.10.11 (tags/v3.10.11:7d4cc5a, Apr 5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]', 'Platform': 'Windows-10-10.0.22631-SP0', 'Platform Version': '10.0.22631', 'Processor': 'Intel64 Family 6 Model 158 Stepping 10, GenuineIntel', 'GPU': "_CudaDeviceProperties(name='NVIDIA GeForce GTX 1650', major=7, minor=5, total_memory=4095MB, multi_processor_count=16)", 'Cuda Version': '12.4', 'model_id': 'facebook/nllb-200-distilled-600M'}

• Serving Flask app 'nllb_serve.app'
• Debug mode: off INFO:werkzeug:WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
• Running on all addresses (0.0.0.0)
• Running on http://127.0.0.1:6060
• Running on http://192.168.0.106:6060 INFO:werkzeug:Press CTRL+C to quit INFO:root:Loading tokenizer for facebook/nllb-200-distilled-600M; src_lang=jpn_Jpan ... ERROR:nllb_serve.app:Exception on /translate [POST] Traceback (most recent call last): File "C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\flask\app.py", line 1463, in wsgi_app response = self.full_dispatch_request() File "C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\flask\app.py", line 872, in full_dispatch_request rv = self.handle_user_exception(e) File "C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\flask\app.py", line 870, in full_dispatch_request rv = self.dispatch_request() File "C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\flask\app.py", line 855, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(view_args) # type: ignore[no-any-return] File "C:\Windows\System32\nllb-serve\nllb_serve\app.py", line 145, in translate inputs, forced_bos_token_id=tokenizer.lang_code_to_id[tgt_lang], AttributeError: 'NllbTokenizerFast' object has no attribute 'lang_code_to_id' INFO:werkzeug:127.0.0.1 - - [24/Sep/2024 20:05:00] "POST /translate HTTP/1.1" 500 -

Tried to find Solution Online and on ChatGPT..

ChatGPT: The error you're encountering indicates that the NllbTokenizerFast object does not have the attribute lang_code_to_id. This typically suggests that the tokenizer you are using is either outdated or improperly initialized.

Here are some steps to troubleshoot and resolve the issue:

1. Update Transformers Library: Ensure that you have the latest version of the transformers library. You can update it using pip:

pip install --upgrade transformers

2. Check Tokenizer Initialization: Make sure you are correctly initializing the tokenizer. It should look something like this:

from transformers import NllbTokenizerFast

tokenizer = NllbTokenizerFast.from_pretrained("facebook/nllb-200-distilled-600M")

3. Accessing Language Codes: Instead of lang_code_to_id, you might need to access language codes differently. You can check available language codes with:

print(tokenizer.get_langs())

4. Update Your Code: If lang_code_to_id is not available, check the documentation for the latest method to get the language IDs or update the way you handle language parameters in your translation request.

5. Check Documentation and Issues: Visit the official Hugging Face documentation or the GitHub repository for any changes in the API or for reported issues related to the NllbTokenizerFast.

If after these steps the error persists, please provide additional context or code snippets for further diagnosis.

[matsbar]

I can confirm that this happens to me too; both in Windows 11 Pro, and Linux Mint 22 Xfce. I suspect that we are too low on RAM, I too have only 4GB, same as you.

[zerofdest]

It has nothing to do with RAM. The true reason is that the version of transformers is too high for NLLB-serve so the solution is: Try downgrading to an older version, such as 4.37.0 pip install transformers==4.37.0