Counterfactuals fail to work

[tanikina]

Here is the stack trace for the following input: "cfe for id 14"

[2023-06-08 09:49:41,263] INFO in core: USER INPUT: cfe for id 14

Batches: 100%|████████████████████████████████████| 1/1 [00:00<00:00, 40.87it/s] [2023-06-08 09:49:41,335] INFO in core: adapters decoded text filter id 14 and cfe [2023-06-08 09:50:46,811] INFO in flask_app: Traceback getting bot response: Traceback (most recent call last): File "/home/ubuntu/projects/InterroLang/actions/perturbation/cfe_generation.py", line 51, in get_samples_from_pj generated_samples = self.explainer.perturb(instance, ctrl_code=ctrl_code, num_perturbations=None, File "/home/ubuntu/miniconda3/envs/nlg/lib/python3.9/site-packages/polyjuice/polyjuice_wrapper.py", line 263, in perturb ctrl = self.detect_ctrl_code(orig_doc, generated_doc, eop) File "/home/ubuntu/miniconda3/envs/nlg/lib/python3.9/site-packages/polyjuice/polyjuice_wrapper.py", line 131, in detect_ctrl_code meta.compute_metadata( File "/home/ubuntu/miniconda3/envs/nlg/lib/python3.9/site-packages/polyjuice/compute_perturbs/compute_ctrl_meta.py", line 546, in compute_metadata p.compute_metadata(sentence_similarity) File "/home/ubuntu/miniconda3/envs/nlg/lib/python3.9/site-packages/polyjuice/compute_perturbs/compute_ctrl_meta.py", line 386, in compute_metadata negs2, neg_heads2 = get_negations(bcore) File "/home/ubuntu/miniconda3/envs/nlg/lib/python3.9/site-packages/polyjuice/compute_perturbs/compute_ctrl_meta.py", line 156, in get_negations for t in span: TypeError: 'NoneType' object is not iterable

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/ubuntu/projects/InterroLang/flask_app.py", line 190, in get_bot_response response = BOT.update_state(user_text, conversation) File "/home/ubuntu/projects/InterroLang/logic/core.py", line 892, in update_state returned_item = run_action( File "/home/ubuntu/projects/InterroLang/logic/action.py", line 48, in run_action action_return, action_status = actions[p_text]( File "/home/ubuntu/projects/InterroLang/actions/perturbation/cfe.py", line 80, in counterfactuals_operation same, diff = cfe_explainer.cfe(instance, cfe_num, ctrl_code=ALL_CTRL_CODES, _id=_id) File "/home/ubuntu/projects/InterroLang/actions/perturbation/cfe_generation.py", line 67, in cfe new_samples = self.get_samples_from_pj(instance, ctrl_code) File "/home/ubuntu/projects/InterroLang/actions/perturbation/cfe_generation.py", line 54, in get_samples_from_pj generated_samples = self.explainer.perturb(instance, ctrl_code=ALL_CTRL_CODES, num_perturbations=None, File "/home/ubuntu/miniconda3/envs/nlg/lib/python3.9/site-packages/polyjuice/polyjuice_wrapper.py", line 263, in perturb ctrl = self.detect_ctrl_code(orig_doc, generated_doc, eop) File "/home/ubuntu/miniconda3/envs/nlg/lib/python3.9/site-packages/polyjuice/polyjuice_wrapper.py", line 131, in detect_ctrl_code meta.compute_metadata( File "/home/ubuntu/miniconda3/envs/nlg/lib/python3.9/site-packages/polyjuice/compute_perturbs/compute_ctrl_meta.py", line 546, in compute_metadata p.compute_metadata(sentence_similarity) File "/home/ubuntu/miniconda3/envs/nlg/lib/python3.9/site-packages/polyjuice/compute_perturbs/compute_ctrl_meta.py", line 386, in compute_metadata negs2, neg_heads2 = get_negations(bcore) File "/home/ubuntu/miniconda3/envs/nlg/lib/python3.9/site-packages/polyjuice/compute_perturbs/compute_ctrl_meta.py", line 156, in get_negations for t in span: TypeError: 'NoneType' object is not iterable

[2023-06-08 09:50:46,811] INFO in flask_app: Exception getting bot response: 'NoneType' object is not iterable

[tanikina]

Possible work-around (w/o PolyJuice): https://github.com/nfelnlp/InterroLang/commit/50b959bda7e52f6f3da2fcfa47b35a612d986a83

[qiaw99]

This error seems to be caused within polyjuice package. I also get the same error.

[qiaw99]

When I use other id instance, it still works... Not sure what's happening in polyjuice

[tanikina]

Seems to be an issue specific for id 14 (PolyJuice-internal).