Develop mechanism for translating Spanish tweets to English.

MElkamhawy / PlaceboAffect

This repo is used for our team PlaceboAffect for LING 573 Course at UW Seattle.

Apache License 2.0

6 stars 0 forks source link

Develop mechanism for translating Spanish tweets to English. #53

Closed kharaldsson closed 1 year ago

alex-321 commented 1 year ago

Packages and models tried:

googletrans
- got ProtocolError: Invalid input ConnectionInputs.RECV_PING in state ConnectionState.CLOSED when translating dev set
deep_translator
- 6.5 min, but also 11.5 min on 500 tweets (dev set)
TextBlob
- 40s, but also 4.5 min on 500 tweets (dev set)
Opus MT
- Was taking a very long time

kharaldsson commented 1 year ago

Let's discuss it tomorrow. I wonder whether we just use textblob. That could be the extent of our adaptation. Translate everything to english. Train on the combined en and es training set (post-translation). Perhaps even pre-translate test set.

kharaldsson commented 1 year ago

Per discussion today, @alex-321 will develop a script that takes a Spanish file and outputs a deep_translator translated version of it. Such files will be written to data/train/es2en, data/dev/es2en, data/test/es2en.

alex-321 commented 1 year ago

FYSA running via CLI:

dev (500): 1:38
test (1600): 16:25
train (4500): 1:07:14