marcotcr / checklist

Beyond Accuracy: Behavioral Testing of NLP models with CheckList
MIT License
2.01k stars 204 forks source link

Could you give a more detailed environment? #17

Closed cutelittletiantian closed 4 years ago

cutelittletiantian commented 4 years ago

There is much troubleshooting when following your guide in README.md, none of the demo codes can be replicated on my computer. I don't know if there are any hidden bugs or it's only for my issue. Then could you give us a more detailed environment, including the libraries needed and their version?

marcotcr commented 4 years ago

Did you follow the installation instructions here? It should install all of the right packages.

cutelittletiantian commented 4 years ago

Yes, of course, I have done it! I have tried many times in different ways. I used python 3.6.5 and I'm not quite sure about its compatibility though. I can only run quite a few of the codes in Jupyter Notebook.

marcotcr commented 4 years ago

What error are you getting?

cutelittletiantian commented 4 years ago

Mostly there are the 'BadZipFile: File is not a zip file' errors. I think probably there is some problem with what I download inside but I can't spot it.

For example:

import checklist
from checklist.editor import Editor
from checklist.perturb import Perturb
from checklist.test_types import MFT, INV, DIR
editor = Editor()

t = editor.template('This is {a:adj} {mask}.',  
                      adj=['good', 'great', 'excellent', 'awesome'])
test1 = MFT(t.data, labels=1, name='Simple positives',
           capability='Vocabulary', description='')

This is what you provide with us in the first cell of a jupyter file. Then here's the error I got:

---------------------------------------------------------------------------
BadZipFile                                Traceback (most recent call last)
<ipython-input-7-8d49cc763522> in <module>
      1 import checklist
      2 from checklist.editor import Editor
----> 3 from checklist.perturb import Perturb
      4 from checklist.test_types import MFT, INV, DIR
      5 editor = Editor()

~\anaconda3\envs\checklist\lib\site-packages\checklist\perturb.py in <module>
      5 import json
      6 import pattern
----> 7 from pattern.en import tenses
      8 from .editor import recursive_apply, MunchWithAdd
      9 

~\anaconda3\envs\checklist\lib\site-packages\pattern\text\en\__init__.py in <module>
     77 )
     78 # Import all submodules.
---> 79 from pattern.text.en import inflect
     80 from pattern.text.en import wordnet
     81 from pattern.text.en import wordlist

~\anaconda3\envs\checklist\lib\site-packages\pattern\text\en\__init__.py in <module>
     78 # Import all submodules.
     79 from pattern.text.en import inflect
---> 80 from pattern.text.en import wordnet
     81 from pattern.text.en import wordlist
     82 

~\anaconda3\envs\checklist\lib\site-packages\pattern\text\en\wordnet\__init__.py in <module>
     55 for token in ("wordnet", "wordnet_ic", "sentiwordnet"):
     56     try:
---> 57         nltk.data.find("corpora/" + token)
     58     except LookupError:
     59         try:

~\anaconda3\envs\checklist\lib\site-packages\nltk\data.py in find(resource_name, paths)
    555             modified_name = "/".join(pieces[:i] + [pieces[i] + ".zip"] + pieces[i:])
    556             try:
--> 557                 return find(modified_name, paths)
    558             except LookupError:
    559                 pass

~\anaconda3\envs\checklist\lib\site-packages\nltk\data.py in find(resource_name, paths)
    542                 if os.path.exists(p):
    543                     try:
--> 544                         return ZipFilePathPointer(p, zipentry)
    545                     except IOError:
    546                         # resource not in zipfile

~\anaconda3\envs\checklist\lib\site-packages\nltk\compat.py in _decorator(*args, **kwargs)
     39     def _decorator(*args, **kwargs):
     40         args = (args[0], add_py3_data(args[1])) + args[2:]
---> 41         return init_func(*args, **kwargs)
     42 
     43     return wraps(init_func)(_decorator)

~\anaconda3\envs\checklist\lib\site-packages\nltk\data.py in __init__(self, zipfile, entry)
    394         """
    395         if isinstance(zipfile, str):
--> 396             zipfile = OpenOnDemandZipFile(os.path.abspath(zipfile))
    397 
    398         # Check that the entry exists:

~\anaconda3\envs\checklist\lib\site-packages\nltk\compat.py in _decorator(*args, **kwargs)
     39     def _decorator(*args, **kwargs):
     40         args = (args[0], add_py3_data(args[1])) + args[2:]
---> 41         return init_func(*args, **kwargs)
     42 
     43     return wraps(init_func)(_decorator)

~\anaconda3\envs\checklist\lib\site-packages\nltk\data.py in __init__(self, filename)
    934         if not isinstance(filename, str):
    935             raise TypeError("ReopenableZipFile filename must be a string")
--> 936         zipfile.ZipFile.__init__(self, filename)
    937         assert self.filename == filename
    938         self.close()

~\anaconda3\envs\checklist\lib\zipfile.py in __init__(self, file, mode, compression, allowZip64)
   1106         try:
   1107             if mode == 'r':
-> 1108                 self._RealGetContents()
   1109             elif mode in ('w', 'x'):
   1110                 # set the modified flag so central directory gets written

~\anaconda3\envs\checklist\lib\zipfile.py in _RealGetContents(self)
   1173             raise BadZipFile("File is not a zip file")
   1174         if not endrec:
-> 1175             raise BadZipFile("File is not a zip file")
   1176         if self.debug > 1:
   1177             print(endrec)

BadZipFile: File is not a zip file

Similar errors always occurred in other files whenever I tried to import those libraries.

marcotcr commented 4 years ago

You have a problem with the pattern package. Check this out.

cutelittletiantian commented 4 years ago

Great, I can run them again. Thank you!