search

DFKI-NLP / InterroLang

InterroLang: Exploring NLP Models and Datasets through Dialogue-based Explanations [EMNLP 2023 Findings]
https://arxiv.org/abs/2310.05592
5 stars 1 forks source link

[Operations] similar and rationalize operations for OLID and DailyDialog #56

Closed nfelnlp closed 1 year ago

nfelnlp commented 1 year ago

In order to complete the prototypes for OLID and DailyDialog, we need to implement adaptations for the operations similar and rationalize.

OLID

DailyDialog

nfelnlp commented 1 year ago

OLID option added for rationalize : https://github.com/nfelnlp/InterroLang/commit/12fc322e9401ebb61ea87998ff5317e9f27abed4

The OLID rationales with GPT-Neo-2.7B were quite poor in quality, unfortunately. As a test, I exchanged it with one of the newer LLMs, dolly-v2-3b and got very promising results:

Screenshot from 2023-05-20 15-02-39

Since dolly-v2-3b has the same base model (GPTNeoX), we could easily also analyze the parsing accuracy for that one.

nfelnlp commented 1 year ago

similar solved by #70