CODAIT / text-extensions-for-pandas

Natural language processing support for Pandas dataframes.
Apache License 2.0
217 stars 34 forks source link

Stabilize tests that rely on SpaCy models #47

Closed frreiss closed 4 years ago

frreiss commented 4 years ago

Some of our regression tests keep failing because the SpaCy models that they depend on keep changing.

We need to adjust our build/test dependencies so that the SpaCy model versions that we run regression tests against do not change from day to day.

Example of failing tests:

$ python -m unittest discover
Downloading: 100%|███████████████████████████| 232k/232k [00:00<00:00, 5.72MB/s]
.......
======================================================================
FAIL: test_iob_to_spans (text_extensions_for_pandas.io.test_conll.CoNLLTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/CODAIT/text-extensions-for-pandas/text_extensions_for_pandas/io/test_conll.py", line 57, in test_iob_to_spans
    4  [100, 113): 'Steven Wright'   PERSON"""
AssertionError: "    [45 chars]    [61, 67): 'Alaska'      GPE\n1      [73, 8[63 chars]RSON" != "    [45 chars]    [4, 11): 'Bermuda'      ORG\n1         [12[145 chars]RSON"
                      token_span ent_type
+ 0           [4, 11): 'Bermuda'      ORG
+ 1         [12, 20): 'Triangle'  PRODUCT
- 0           [61, 67): 'Alaska'      GPE
? ^
+ 2           [61, 67): 'Alaska'      GPE
? ^
- 1      [73, 84): 'Santa Claus'   PERSON
? ^
+ 3      [73, 84): 'Santa Claus'   PERSON
? ^
- 2  [100, 113): 'Steven Wright'   PERSON? ^
+ 4  [100, 113): 'Steven Wright'   PERSON? ^
======================================================================
FAIL: test_make_tokens_and_features (text_extensions_for_pandas.io.test_spacy.IOTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/CODAIT/text-extensions-for-pandas/text_extensions_for_pandas/io/test_spacy.py", line 71, in test_make_tokens_and_features
    (8, 8, [34, 35): '.', [34, 35): '.', '.', 'PUNCT', '.', 'punct', 1, '.', 'O', '', False, False, [0, 35): 'She sold c shills by the Sith Lord.')]"""
AssertionError: "[(0,[355 chars]N', 'det', 3, 'x', 'O', '',  True, False, [0, [970 chars].')]" != "[(0,[355 chars]N', 'compound', 3, 'x', 'O', '',  True, False,[975 chars].')]"
Diff is 1957 characters long. Set self.maxDiff to None to see it.
----------------------------------------------------------------------
Ran 89 tests in 1.210s
FAILED (failures=2, skipped=1)
The command "python -m unittest discover" exited with 1.
frreiss commented 4 years ago

Tests seem to be stable now after #58, which pinned the SpaCy dependency to a specific version.