search

marcotcr / checklist

Beyond Accuracy: Behavioral Testing of NLP models with CheckList
MIT License
2.01k stars 204 forks source link

Where is the processed_qqp.pkl in the qqp notebook #12

Closed yyinsomnia closed 4 years ago

yyinsomnia commented 4 years ago

Where is the spacy_map = pickle.load(open('/home/marcotcr/tmp/processed_qqp.pkl', 'rb'))

marcotcr commented 4 years ago

I guess I cached the spacy-processed questions rather than computing them in the notebook. You can process the questions yourself if you replace that line with

all_questions = list(all_questions)
parsed_questions = list(nlp.pipe(all_questions))
spacy_map = dict([(x, y) for x, y in zip(all_questions, parsed_questions)])

I updated the notebook to reflect this in 64a810a