What is spacy_map = pickle.load(open('/home/marcotcr/tmp/processed_squad.pkl', 'rb'))

marcotcr / checklist

Beyond Accuracy: Behavioral Testing of NLP models with CheckList

MIT License

2.01k stars 204 forks source link

What is spacy_map = pickle.load(open('/home/marcotcr/tmp/processed_squad.pkl', 'rb')) #66

Closed kingafy closed 3 years ago

kingafy commented 3 years ago

What information is lying in the processed_squad.pkl file?I want to create a custom suite for Question answering on evaluation of a domain specific data.Can you elaborate what lies in processed_squad.pkl file and the real objective of using this in the code?

marcotcr commented 3 years ago

This is just a map of string to spacy doc, to avoid having to run spacy every time I run the notebook. You can ignore it and run spacy on your own data.