search

cs50 / check50

This is check50, a command-line program with which you can check the correctness of your programs.
GNU General Public License v3.0
393 stars 263 forks source link

PicklingError when importing cs50 library in checks #235

Open kzidane opened 3 years ago

kzidane commented 3 years ago 
Traceback (most recent call last):
  File "/usr/local/bin/check50", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/dist-packages/check50/__main__.py", line 379, in main
    check_results = check_runner.run(args.target)
  File "/usr/local/lib/python3.7/dist-packages/check50/runner.py", line 201, in run
    result, state = future.result()
  File "/usr/lib/python3.7/concurrent/futures/_base.py", line 428, in result
    return self.__get_result()
  File "/usr/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/lib/python3.7/multiprocessing/queues.py", line 236, in _feed
    obj = _ForkingPickler.dumps(obj)
  File "/usr/lib/python3.7/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function <lambda> at 0x7ff58294add0>: attribute lookup <lambda> on cs50.cs50 failed
Jelleas commented 3 years ago

Do you have the relevant slug by any chance?

kzidane commented 3 years ago

cs50/problems/2020/fall/movies

TheRijn commented 3 years ago

Seems to be fixed in the online version of check50, with -l this error still exists.

Jelleas commented 3 years ago

This happens because the cs50 library sets the sys.excepthook itself unconditionally to a lambda function. That famously are not pickle-able, because where'd be the fun in that.

As it happens, check50 now treats sys.excepthook as something special, because from Python 3.8 on OS X check50 is forced to support the spawn method for multiprocessing. Long story short, this line: https://github.com/cs50/check50/blob/03d5bb037c9323b8ab9d45cdd4d54613a2f8d924/check50/runner.py#L333 now causes check50 to copy w/e excepthook is set to each check process.

In part this problem stems from the awkwardness of a library changing sys.excepthook. No one expects tracebacks to change thanks to this import:

from cs50 import SQL

I think the only approach here is to accept that sys.excepthook can be set externally, either by the checks file, or a library such as python-cs50. If that happens then either under spawn pass nothing, because that imported library should be imported again and set sys.excepthook again. Or under fork don't set sys.excepthook.

Until then, either move from cs50 import SQL from the top of the test file to each test/function where it's needed. Moving that import to line 119 fixes movies for instance.

Jelleas commented 3 years ago

https://github.com/cs50/check50/tree/unpickleable_attributes