Open evilc3 opened 1 year ago
anyone?
Can confirm a similar issue. Here's our code:
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("facebook/nllb-200-distilled-600M",
use_auth_token=False, local_files_only=True, src_lang="spa_Latn")
model = AutoModelForSeq2SeqLM.from_pretrained("facebook/nllb-200-distilled-600M",
use_auth_token=False, local_files_only=True)
article = "123"
inputs = tokenizer(article, return_tensors="pt")
translated_tokens = model.generate(
**inputs, forced_bos_token_id=tokenizer.encode("eng_Latn")[1], max_length=30)
output = tokenizer.batch_decode(translated_tokens, skip_special_tokens=True)[0]
print(output)
Output:
The Commission shall adopt implementing acts.
When changing to src_lang="tgk_Cyrl"
the output is 123 What is the meaning of life?
Incorrect translation when translating English to spanish.
When source input contains only numbers, no letters the translated output is completely incorrect. please see the examples below.
Model Used : NLLB-600m distilled.
Predicted Output
For input
This problem seems to persist when using bigger model.
Steps to reproduce the behavior
Code sample
Environment
Possible corrections