search

fe1ixxu / ALMA

State-of-the-art LLM-based translation models.
MIT License
395 stars 29 forks source link

Question on cpo loss #36

Closed vince62s closed 6 months ago

vince62s commented 6 months ago

I did not try to reproduce the training process but I have a question on the paper and the code.

when you add up cpo_loss and clm_loss what are the orders? I mean are both component about the same value is one really smaller than the other one ?

thanks.

fe1ixxu commented 6 months ago

It depends on the tasks. But for machine translation, they are in the similar order.