Unable to run inference after finetuning

[allincowell]

Hi Pranjal,

I am not able to run inference after finetuning as the checkpoints do not seem to have vocab directory. How to work around it then?

Thanks, Rahul

[PranjalChitale]

Copy the vocab directory from the base model you fine-tuned.

[allincowell]

I am still getting an error like: "FileNotFoundError: [Errno 2] No such file or directory: './indic-en-exp/model_pt/final_bin/dict.SRC.txt'"

I am running inference.engine code and ckpt_dir contains .pt (finetuned checkpoint) and vocab directory.

[PranjalChitale]

Copy the final_bin folder (which contains these dicts) from the base model you fine-tuned.

[allincowell]

I am able to run it now. But got 14 BLEU on validation while finetuning but doing inference and evaluation I am getting 0.5 BLEU on validation itself. I am these command now: "bash joint_translate.sh " and "bash compute_metrics.sh ". I copied the dictionaries accordingly.

[jaygala24]

Hey @allincowell

The arguments that you are passing to the joint_translate.sh script seem to be incorrect as the script takes 5 arguments. Please read the highlighted section for further details on the arguments and its intended order. Thank you!

[allincowell]

Actually, this was the command I was using: bash joint_translate.sh indic-en-exp/test/test.san_Deva indic-en-exp/test/test_output.hin_Deva san_Deva hin_Deva indic-en-exp/model_pt

This one has 5 arguments in correct order.

[PranjalChitale]

Just to confirm, you seem to be trying Indic-Indic evaluation (using a Indic-En checkpoint?) or are the directory names saved like that ?

Please confirm your experimental config once to check whether you are using the appropriate model (Indic-Indic) for the appropriate translation direction.

[allincowell]

The directory is named just like that. I am using indic-indic checkpoint and my dataset is Prakrit-Hindi in Devanagiri script.

[PranjalChitale]

Which language tag did you use during FT ?

Is the same tag being used during eval ?

What was your fine-tuning data mixture ?

[allincowell]

I used san_Deva for Prakrit and hin_Deva for Hindi. I am using the same for evaluation as well. The original sentences were in Prakrit. I can see good translations made for validation dataset but when I do inference on validation dataset, the quality is really bad.

I have also used the vocab and dictionary as the one which was with the original pretrained checkpoint.

[datha29]

Hi Pranjal,

I am not able to run inference after finetuning as the checkpoints do not seem to have vocab directory. How to work around it then?

Thanks, Rahul

Hi Rahul

Can you please guide how you went about for finetuning.I mean iif any meta data used & whether the model is fine tuned

Regards

Datha

[PranjalChitale]

@allincowell

Closing this due to inactivity, feel free to reopen if the issue has not been resolved yet.