Hi, I am testing this code on a list of 5 short sentences, the average time for translation is 2 seconds/sentence. which is slow for my requirements. any hints on how to speed-up the translation ? Thanks
import dl_translate as dlt
import time
french_sentence = 'oh mon dieu c mechant c pas possible jamais je reviendrai, a deconseiller. je vous recommende de visiter un autre produit apres vous pouvez voire la difference'
arabic_sentence = ' لقد جربت عدة نسخ من هذا المنتج لكن لم استطع ان اجد فبه ما ينتج ما هذا الهراء'
ar2 = 'المنتج الاصلى سريع الذوبان فى الماء ويذوب بشكل مثالى على عكس المكمل المغشوش ...منتج كويس انا حبيتو و بنصح فيه'
ar3= 'امشي سيدا لفه الثانيه يسار تعدد المطالبات المتعلقة بالأراضي وما ينتج عن ذلك من تناحر يولد باستمرار نزاعات متجددة. ... ويمكن دمج ما ينتج عن ذلك من معارف في إطار برنامج عمل نيروبي'
nepali ='यो मृत्युदर विकासशील देशहरुमा धेरै छ'
sent_list =[french_sentence, arabic_sentence, ar2, ar3, nepali]
print(sent_list)
mt = dlt.TranslationModel() # Slow when you load it for the first time
map_langdetect_to_translate = {'ar':'Arabic', 'en':'English', 'es':'Spanish', 'fr':'French', 'ne':'Nepali'}
start = time.time()
for sent in sent_list:
print('-------------------------------------')
print('original sentence is : ',sent)
print('detected lang ',detect(sent))
mapped = map_langdetect_to_translate[detect(sent)]
translated = mt.translate(sent, source=mapped, target="en")
print('Translation is : ',translated)
end = time.time()
tt = time.strftime("%H:%M:%S", time.gmtime(end-start))
time_message = 'Query execution time : {}'.format( tt )
print(time_message)
Use a GPU when it's available, otherwise use as many CPU cores as possible
Try to batch the same languages together (in your case, the arabic sentences can be passed as a single list to mt.translate), since the translation processes in batches
Try to use mbart50 and see if it might be faster/slower
If you have a very long sequence, you can try to break it down into multiple sentences
Hi, I am testing this code on a list of 5 short sentences, the average time for translation is 2 seconds/sentence. which is slow for my requirements. any hints on how to speed-up the translation ? Thanks