Revision I

[CahidArda]

The doc issue

Things to do:

Karahan:

• [x] Go over the paper to improve ambiguities in expressions (by noting down the changes)
  • Listed all my changes in a Notion page, gonna share them soon.
• [x] Twitter sentiment analysis
• [x] Compare results with Finnish, Hungarian
  • Added a section to the Conclusion where we compare our results with the results of another word embedding comparison study in Finnish. Similar to ours, GloVe lags behind Word2Vec and FastText in general (despite its performance in English). Review required.
• [x] Innovation issue: Add section on how this is the first paper to do a comprehensive study for Turkish
  • Added the related section to the end of "Related Works," mainly discussing how we incorporate all the state-of-the-art static word embedding models with a huge amount of data for evaluation. Then, we pointed out the novel finding that X2Static BERT outperforms traditional word embeddings, a topic not previously addressed for Turkish. Review required.
• [x] Comparing decontextualized elmo with paper

Arda:

• [x] Go over the paper to improve ambiguities in expressions (by noting down the changes)
• [ ] ~Increase the number of models (GPT possible? If not possible write a section. Counterargument -> cost, find paper that shows word2vec, glove and fasttext?)~
• [x] twitter x2static (add to tag), bert
• [x] Compare results with English Paper
• [x] Add conclusions for other languages like the paper shared with us
• [x] Implication: Add real life examples like how using a better model will improve accuracy

typos:

• [x] using enquote
• [x] Removing duplicate names
• [x] remove unnecessary et. al

If we have time:

• [x] Image (png to pdf)
• [ ] ~GPT~
• [x] Multiple runs and confidence intervals for extrinsic evaluations

Errand:

• [x] Update the Remote Server Configuration section to either include a new link for turkish-text-tokenized.txt or remove the second solution altogether.

Important: ESwA requires us to submit all the tables individually as well. Don't forget to update the relevant tables and add references if necessary (e.g. for the new Turkish Twitter Dataset)

Suggest a potential alternative/fix

No response

[KarahanS]

Decontextualized simply stands for "aggregated" - which is actually one of the decontextualization methods.

[KarahanS]

Slightly updated the settings for Sentiment Analysis.

For the sentiment analysis tasks, we configured the maximum number of epochs to 15. Additionally, we implemented an early stopping criterion: if the error on the validation set begins to increase, we halt the training phase.

To be consistent among the NLP tasks in terms of the number of epochs to train, I believe it is better to increase the number of epochs from 5 to 15 with an early stopping criterion. As a result of this update, we have to run X2Static BERT not only for the Twitter Dataset but for all of the three NLP tasks, setting the random seed = 7 with 15 epochs.

[KarahanS]

Updated the NLP code. Provide the path of the X2Static model:

    "x2_bert": {
        "model": os.path.join(FOLDER, ""),
        "dim": 768,
        "binary": False,
        "no_header": False,
    },

Then you can run the code with the following command for the third dataset (number of epochs = 15 with early stopping):

 python sentiment.py -d 3 -e 15 -w x2_bert 

[CahidArda]

added section for generalizing to other languages. Added examples of word embedding usage in the conclusion section

[KarahanS]

Successfully passed the Revision I, closing the issue.