A TalkToModel (Slack et al., 2022) adaptation to NLP use cases (question answering, hate speech detection, dialogue act classification).
The name is a word-play on Interrobang, a ligature of question mark and exclamation mark, and the interrogation of Language models.
Our tool offers a dialogue-based exploration of NLP interpretability methods (feature attribution, counterfactuals and perturbations, free-text rationalization) and dataset analyses (similar examples, keywords, label distribution).
Our accompanying paper is accepted at EMNLP 2023 Findings.
We consider 7 categories of operations.
They are defined in actions and prompts.
We provide a dataset view with which users can explore instances contained in the pre-defined dataset (in screenshot, BoolQ dataset is used). Users can search instances that include the entered string.
conda create -n interrolang python=3.9
conda activate interrolang
python -m venv venv
source venv/venv/activate
python -m pip install --upgrade pip
pip install -r requirements.txt
# Download omw-1.4
python -m nltk.downloader omw-1.4
# Punkt
python -m nltk.downloader punkt
# Install spacy
pip install -U pip setuptools wheel
pip install -U spacy
python -m spacy download en_core_web_sm
Install polyjuice-nlp and its dependencies due to some issues from polyjuice:
cd utils
bash dependency.sh
In our tool, we currently use the following Transformer models:
Put them under ./data
and name the folders boolq_model
and olid_model
respectively.
Put the file 5e_5e-06lr
under ./explained_models/da_classifier/saved_model
In ./configs
, there are all gin config files for all three datasets with different parsing models. You can choose one of them and set its path in ./global_config.gin
:
GlobalArgs.config = "./configs/boolq_adapter.gin"
You can launch the Flask web app via
python flask_app.py
If you want to run with Docker, you can build the docker app
sudo docker build -t interrolang .
And then run the image
sudo docker run -d -p 4000:4000 interrolang
After the project is set up, we provide an optional user guide using Selenium to demonstrate how to use our system if you have a Chrome Browser available.
Include operation works similar to custom input
@inproceedings{feldhus-etal-2023-interrolang,
title = "{I}nterro{L}ang: Exploring {NLP} Models and Datasets through Dialogue-based Explanations",
author = {Feldhus, Nils and Wang, Qianli and Anikina, Tatiana and Chopra, Sahil and Oguz, Cennet and M{\"o}ller, Sebastian},
booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2023",
month = dec,
year = "2023",
address = "Singapore, Singapore",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/2310.05592",
}