Wrong translation for a Hindi Sentence using Indic-En Model

[harshith-77]

I am Encountering an issue when I try to translate the sentence "आप कैसे हैं?" from Hindi to English, it is being wrongly translated to "कैसा हैं?" when using Indic-En Model (ai4bharat/indictrans2-indic-en-1B). The below image is a screenshot of the output produced in the colab notebook you have provided in this repo.

But when I am trying to translate the same sentence in https://ai4bharat.iitm.ac.in/resources/models/ website, expected output is being generated

The thing is that other than this sentence all the others are being translated correctly. What might be the reason causing this issue and what can I do to solve it?

[prajdabre]

What command did you use?

[harshith-77]

I have used the colab provided in this repo. I didn't change anything except the input sentence in Indic to English Example section as shown below.

[prajdabre]

Hi

So I tested the notebook and believe it or not, I think "Aap kaise hai?" (In devanagari) Is a cursed example. Not sure why but this is the one example where the model fails. The moment you change it to "Raj Aap kaise hai?" And it works perfectly.

I don't think we can do anything about this. Alternatively can you try the distilled model once?

[tinku2603]

Yes, When we add some prefix to this sentence, it's translating perfectly.

[harshith-77]

Same issue with "आप कैसा है" and "तुम कैसे हो". Anyways I will try with distilled model as well.