Pickle failure is raising AttributeError and not PicklingError

ea6bafcc-e6c0-4b72-ac85-c8610de892d5 commented 7 years ago

BPO	29187
Nosy	@avassalotti, @serhiy-storchaka, @iritkatriel

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = None created_at = labels = ['extension-modules', 'type-bug', '3.8', '3.9', '3.10'] title = 'Pickle failure is raising AttributeError and not PicklingError' updated_at = user = 'https://bugs.python.org/MattDodge' ``` bugs.python.org fields: ```python activity = actor = 'iritkatriel' assignee = 'none' closed = False closed_date = None closer = None components = ['Extension Modules'] creation = creator = 'Matt.Dodge' dependencies = [] files = [] hgrepos = [] issue_num = 29187 keywords = [] message_count = 3.0 messages = ['284869', '284896', '387185'] nosy_count = 4.0 nosy_names = ['alexandre.vassalotti', 'serhiy.storchaka', 'Matt.Dodge', 'iritkatriel'] pr_nums = [] priority = 'normal' resolution = None stage = 'patch review' status = 'open' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue29187' versions = ['Python 3.8', 'Python 3.9', 'Python 3.10'] ```

ea6bafcc-e6c0-4b72-ac85-c8610de892d5 commented 7 years ago

When failing to pickle something (like a locally scoped class) the documentation indicates that a PicklingError should be raised. Doc links:

However, instead I'm seeing AttributeError get raised instead, starting in Python 3.5. In Python 3.4 PicklingErrror was raised, as expected.

To reproduce, use the following file \<pickletest.py> def func(): class C: pass return C import pickle pickle.dumps(func()())

In Python 3.4 you see:
Traceback (most recent call last):
  File "pickletest.py", line 5, in <module>
    pickle.dumps(func()())
_pickle.PicklingError: Can't pickle <class '__main__.func.<locals>.C'>: attribute lookup C on __main__ failed

But in 3.5/3.6 you see:
Traceback (most recent call last):
  File "pickletest.py", line 5, in <module>
    pickle.dumps(func()())
AttributeError: Can't pickle local object 'func.<locals>.C'

I don't necessarily mind that a different exception is being raised, but how should we be handling exceptions while pickling? Catch all exceptions? That doesn't feel right to me, but if we're trying to pickle data out of our control I'm not sure what else to do.

FYI, the UnpicklingError documentation (https://docs.python.org/3/library/pickle.html?highlight=pickle#pickle.UnpicklingError) indicates that other exceptions can possibly be raised during unpickling. I assume that is more related to the fact that the pickled data may not fit into the current class/module structure though, so I think it's unrelated.

serhiy-storchaka commented 7 years ago

Python implementation of pickle still raises PicklingError. Seems this was not intentional change.

>>> pickle._dumps(func()())
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 918, in save_global
    obj2, parent = _getattribute(module, name)
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 266, in _getattribute
    .format(name, obj))
AttributeError: Can't get local attribute 'func.<locals>.C' on <function func at 0xb7118d1c>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 1544, in _dumps
    _Pickler(f, protocol, fix_imports=fix_imports).dump(obj)
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 409, in dump
    self.save(obj)
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 521, in save
    self.save_reduce(obj=obj, *rv)
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 605, in save_reduce
    save(cls)
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 978, in save_type
    return self.save_global(obj)
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 922, in save_global
    (obj, module_name, name))
_pickle.PicklingError: Can't pickle <class '__main__.func.<locals>.C'>: it's not found as __main__.func.<locals>.C

iritkatriel commented 3 years ago

Still the same in 3.10:

Python 3.10.0a5+ (heads/bpo-43146-dirty:8f5cf4d381, Feb 17 2021, 14:51:27) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> try:
...     def func():
...         class C: pass
...         return C
...     import pickle
...     pickle.dumps(func()())
... except BaseException as e:
...   exc = e
...
>>> exc
AttributeError("Can't pickle local object 'func.<locals>.C'")
>>>

python / cpython

Pickle failure is raising AttributeError and not PicklingError #73373