Reference code for ACL20 paper - NILE : Natural Language Inference with Faithful Natural Language Explanations.
The code was written with, or depends on:
virtualenv -p python3.6 env
source env/bin/activate
pip install -r requirements.txt
dataset_snli
and dataset_mnli
folders.
bash fetch_data.sh
python prepare_train_test.py --dataset snli --create_data --filter_repetitions
python prepare_train_test.py --dataset mnli --create_data --filter_repetitions
bash run_finetune_gpt2m.sh 0 entailment 2
bash run_finetune_gpt2m.sh 0 contradiction 2
bash run_finetune_gpt2m.sh 0 neutral 2
bash run_finetune_gpt2m.sh 0 all 2
bash run_generate_gpt2m.sh 0 entailment <dataset> all <split>
bash run_generate_gpt2m.sh 0 contradiction <dataset> all <split>
bash run_generate_gpt2m.sh 0 neutral <dataset> all <split>
bash run_generate_gpt2m.sh 0 all <dataset> all <split>
Merge generated explanation
python prepare_train_test.py --dataset <dataset> --merge_data --split <split> --input_prefix gpt2_m_
To merge for the single-explanatoin baseline, run
python prepare_train_test.py --dataset <dataset> --merge_data --split <split> --input_prefix gpt2_m_ --merge_single
Train classifiers on the generated explantions, models are saved at saved_clf
.
NILE-PH
bash run_clf.sh 0 <seed> snli independent independent gpt2_m_ train dev _ _ _
bash run_clf.sh 0 <seed> snli aggregate aggregate gpt2_m_ train dev _ _ _
bash run_clf.sh 0 <seed> snli append append gpt2_m_ train dev _ _ _
NILE-NS
bash run_clf.sh 0 <seed> snli instance_independent instance_independent gpt2_m_ train dev _ _ _
bash run_clf.sh 0 <seed> snli instance_aggregate instance_aggregate gpt2_m_ train dev _ _ _
bash run_clf.sh 0 <seed> snli instance_append instance_append gpt2_m_ train dev _ _ _
NILE
bash run_clf.sh 0 <seed> snli instance_independent instance_independent gpt2_m_ train dev sample _ _
bash run_clf.sh 0 <seed> snli instance_aggregate instance_aggregate gpt2_m_ train dev sample _ _
NILE post-hoc
bash run_clf.sh 0 <seed> snli instance instance gpt2_m_ train dev sample _ _
Single-explanation baseline
bash run_clf.sh 0 <seed> snli Explanation_1 Explanation_1 gpt2_m_ train dev sample _ _
bash run_clf.sh 0 <seed> snli <model> <model> gpt2_m_ _ test _ _ <model_path>
Generated explanations on the e-SNLI dev and test sets are present at ./generated_explanations/*.gz
. Please unzip before using (gunzip <filename>
).
The generated explanations are present in the entailment_explanation,contradiction_explanation,neutral_explanation
columns in a csv format.
We are sharing pre-trained label-specific generators. We are also sharing pre-trained classifiers for NILE-PH Append and NILE Independent architectures.
If you use this code, please consider citing:
[1] Sawan Kumar and Partha Talukdar. 2020. NILE : Natural language inference with faithful natural language explanations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8730–8742, Online. Association for Computational Linguistics. [bibtex]
For any clarification, comments, or suggestions please create an issue or contact sawankumar@iisc.ac.in