search

Helsinki-NLP / Opus-MT

Open neural machine translation models and web services
MIT License
574 stars 71 forks source link

`Helsinki-NLP/opus-mt-fr-en` Model hallucinations #89

Closed KickItLikeShika closed 10 months ago

KickItLikeShika commented 11 months ago

I was wondering if anyone has seen some hallucinations from Helsinki-NLP/opus-mt-fr-en? The model's performance in most of the cases is quite decent, but i might see some strange translations and repeated words at some points Screenshot 2023-08-28 at 7 58 16 PM

translate method:

def translate(text, model):
    inputs = tokenizer(text, return_tensors='pt', max_length=512, truncation=True, padding="max_length")
    inputs = {k: v.to(device) for k, v in inputs.items()}
    translation = model.generate(**inputs)
    translated_text = tokenizer.batch_decode(translation, skip_special_tokens=True)
    return translated_text

UPDATE: I have noticed the same problem with other models too.

mo-shahab commented 10 months ago

yep, this happens in many models too. right now i am facing the same issue with hebrew to english translation and have also raised the issue here in this repo. if u happen to find a way that works out then do update in this thread. and i also faced the issue of slow translation. i wonder how the time taken for translation is for you?

KickItLikeShika commented 10 months ago

@mo-shahab yeah i solved the problem, it's mainly a problem in the sampling/decoding, the default sampling approach to all of the models is mainly greedy search, this article is very helpful and will help you learn more on how to sample/decode your generated text https://huggingface.co/blog/how-to-generate