Uhandled TypeError when running Sentiment analysis test suite

[LoganKells]

When I run the sentinment suite I'm getting the following error, which I do not know how to debug. It occurs in line

This is the code I'm using to run.

from checklist.test_suite import TestSuite

def run():
    # Loading the testing suite
    suite_path = 'release_data/sentiment/sentiment_suite_37.pkl'
    suite = TestSuite.from_file(suite_path)

    # Running tests with precomputed bert predictions (replace bert on pred_path with amazon, google,
    # microsoft, or roberta for others):
    pred_path = 'release_data/sentiment/predictions/bert'
    suite.run_from_file(pred_path, overwrite=True)
    suite.summary()  # or suite.visual_summary_table()
    # suite.visual_summary_table()

if __name__ == "__main__":
    run()

This is the traceback to this line.

abstract_test.py", line 131, in update_expect
    self.results.expect_results = self.expect(self)

/Users/logankells/Documents/Projects/checklist/venv/bin/python /Users/logankells/Documents/Projects/checklist/run_analysis.py
Traceback (most recent call last):
  File "/Users/logankells/Documents/Projects/checklist/run_analysis.py", line 18, in <module>
    run()
  File "/Users/logankells/Documents/Projects/checklist/run_analysis.py", line 13, in run
    suite.run_from_file(pred_path, overwrite=True)
  File "/Users/logankells/Documents/Projects/checklist/checklist/test_suite.py", line 253, in run_from_file
    self.run_from_preds_confs(preds, confs, overwrite=overwrite)
  File "/Users/logankells/Documents/Projects/checklist/checklist/test_suite.py", line 224, in run_from_preds_confs
    t.run_from_preds_confs(p, c, overwrite=overwrite)
  File "/Users/logankells/Documents/Projects/checklist/checklist/abstract_test.py", line 312, in run_from_preds_confs
    self.update_expect()
  File "/Users/logankells/Documents/Projects/checklist/checklist/abstract_test.py", line 131, in update_expect
    self.results.expect_results = self.expect(self)
  File "/home/marcotcr/work/checklist/checklist/expect.py", line 55, in expect
  File "/home/marcotcr/work/checklist/checklist/expect.py", line 55, in <listcomp>
  File "/home/marcotcr/work/checklist/checklist/expect.py", line 116, in expect_fn
  File "/home/marcotcr/work/checklist/checklist/expect.py", line 116, in <listcomp>
  File "/home/marcotcr/work/checklist/checklist/expect.py", line 254, in ret_fn
TypeError: 'UH��H�Ҹ
' not supported between instances of 'NoneType' and 'NoneType'

[marcotcr]

Is release_data/sentiment/sentiment_suite_37.pkl a suite you created? If so, it seems like you're trying to use the predictions for another suite (from release_data/sentiment/predictions/bert) to run it?

[LoganKells]

marcotcr,

No it is not. There is an issue where the given sentiment_suite.pkl was created in an older version of python (such as 3.5) which is incompatible with 3.8+. Therefore I had to install Python 3.7, load the .pkl, then save it to a .pkl which is compatible with modern versions of Python.

[marcotcr]

Hm, can you run the suite on Python 3.7? I suspect the pickled expectation functions may not have transferred to 3.8 when you did the pickling

[LoganKells]

What version of Python did you use to create the .pkl?

[marcotcr]

3.6.8, and I've checked that also works on 3.6.9. I haven't tried 3.8

[LoganKells]

It does load w/o the error I saw with python 3.6.8.