DFKI-NLP / InterroLang

InterroLang: Exploring NLP Models and Datasets through Dialogue-based Explanations [EMNLP 2023 Findings]
https://arxiv.org/abs/2310.05592
5 stars 1 forks source link

[Operations] replace operation #74

Open nfelnlp opened 1 year ago

nfelnlp commented 1 year ago

A possible adaptation of the deprecated TTM operation change (actions/perturbation/what_if.py) is replace where any string can be replaced with a custom input.

This would require two custom inputs, in fact. It would modify the temp_dataset. It is also similar to cfe, but here the user has full control of what they would insert in place of the original string from the data.

A major limitation is that even prediction is not easily possible, because the entire dataset would have to be recomputed. However, this might still be relevant for a small temp_dataset, e.g. you filter by "spider" and are left with 13 instances for BoolQ. With this scale, it would certainly be possible to compute predictions and explanations on the fly.