Translation confidence estimation

[sushain97]

From someone:

• Add a mode that colours the resulting translation depending on how reliable it is.
• One possibility would be for language pairs to have different colours depending on WER or something.

[share-with-me]

@sushain97 How does Apertium translate the content? What rules does it follow for MT? Is there a code snippet at server or some reading material for this? We could perhaps develop something that makes a semantic comparison of original and translated text and depending on accuracy, we could color it!

[sushain97]

Take a look at http://xixona.dlsi.ua.es/~fran/apertium2-documentation.pdf.

[share-with-me]

Awesome! Will have a look!

[share-with-me]

@sushain97 Are we visualising this something like the following? How we process is we scan through the tuples returned from the translate() function from the server and depending upon the confidence factor, we color it. This feature is available if the user clicks on Colored Mode in the navbar and makes it enabled first. This is for visualisation idea only. However, does our current model return translation confidences too along with the success data in ajax call?

Bad translation

Good Translation

[sushain97]

@jonorthwash will have ideas here. I'm not certain what the resolution we can have this data at is.

However, does our current model return translation confidences too along with the success data in ajax call?

No. This requires work on the APy side.

[share-with-me]

@sushain97 Ohh! I would start working on APy side as well! We could discuss ideas with @jonorthwash about progressing on this issue! Thanks!

[jonorthwash]

Yeah, we don't have anything implemented for estimating translation accuracy, so you'll need to set up a plan for figuring that out too.

[share-with-me]

@jonorthwash Ohh cool! I will start with the backend today then!