array.arrays are not unpickleable

[birkenfeld]

BPO	1281383
Nosy	@birkenfeld, @rhettinger, @terryjreedy, @josiahcarlson

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = created_at = labels = ['extension-modules', 'invalid'] title = 'array.arrays are not unpickleable' updated_at = user = 'https://github.com/birkenfeld' ``` bugs.python.org fields: ```python activity = actor = 'mbrighton' assignee = 'none' closed = True closed_date = None closer = None components = ['Extension Modules'] creation = creator = 'georg.brandl' dependencies = [] files = [] hgrepos = [] issue_num = 1281383 keywords = [] message_count = 9.0 messages = ['26191', '26192', '26193', '26194', '26195', '26196', '26197', '26198', '26199'] nosy_count = 6.0 nosy_names = ['georg.brandl', 'rhettinger', 'terry.reedy', 'sjmachin', 'josiahcarlson', 'mbrighton'] pr_nums = [] priority = 'high' resolution = 'not a bug' stage = None status = 'closed' superseder = None type = None url = 'https://bugs.python.org/issue1281383' versions = ['Python 2.4'] ```

[birkenfeld]

Credits to John Machin for discovering this.

""" Googling for "pickle array" in comp.lang.python yields old messages that show a PickleError -- plus one message where Alex Martelli writes "I am but an egg" :O) Looks like arrays are NOW (2.4.1) pickleable but not unpickleable -- see below. I appreciate that arrays are inherently not pickleable because of the type code. However: (1) Anyone know why/when the world changed? (2) If we had alternative constructors like array.iarray(contents) in parallel to array.array('i', contents), those objects could be pickled/unpickled -- yes/no?

Cheers, John \====================

Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310
32 bit (Intel)] on 
win32
Type "help", "copyright", "credits" or "license" for
more information.
 >>> import pickle, array
 >>> class Foo(object):
...    pass
...
 >>> foo = Foo()
 >>> foo.ia = array.array('i', [3,2,1])
 >>> foo.ia
array('i', [3, 2, 1])
 >>> s = pickle.dumps(foo, -1)
 >>> bar = pickle.loads(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "C:\Python24\lib\pickle.py", line 1394, in loads
    return Unpickler(file).load()
  File "C:\Python24\lib\pickle.py", line 872, in load
    dispatch[key](self)
  File "C:\Python24\lib\pickle.py", line 1097, in
load_newobj
    obj = cls.__new__(cls, *args)
TypeError: array() takes at least 1 argument (0 given)

=========== """

[rhettinger]

Logged In: YES user_id=80475

In Py2.4, array's became copyable, weak-referencable, and got support for iterator arguments. Real pickle support wasn't added until Py2.5. The above code fragment is a by-product of pickle making an incorrect guess at how to pickle arrays before real pickel support was added. It is not really a bug; rather, it begs for a feature that wasn't added to later.

If it weren't a new feature, I would just backport the 2.5 pickle support.

[22094973-afb1-482d-a589-036f632b74f7]

Logged In: YES user_id=480138

Please fix the bug in Python 2.4: if array objects are not pickleable in 2.4, then pickle and cPickle should raise a PickleError [like they used to in earlier versions] -- instead of guessing wrongly and misleading callers into thinking that the objects can be pickled.

[ef6b2a61-f027-4805-a66a-cde4eee277c3]

Logged In: YES user_id=341410

Raymond, they seem to be asking for Pickle and cPickle to raise an exception when someone attempts to pickle arrays in a future Python 2.4.2 release. I don't think that is a new feature.

As for 2.5, real pickle support seems reasonable.

[rhettinger]

Logged In: YES user_id=80475

I think you're misunderstanding. Direct pickling of arrays does raise a TypeError. It would be nice if it also did as an object attribute; however, I'm not bothered by it enough to spend development time tracing down the issue and then altering otherwise correct Py2.4 code just to generate a prettier message. It is enough for me that the docs do not promise pickling, that a message is generated by a direct attempt to pickle, that the OP's buggy code eventually errors out, and that everything works fine in Py2.5.

I have no objections to someone finding a way to generate a better error message but think the time would better be spent elsewhere.

>> from array import array >> from pickle import dumps, loads >> ia = array('i', [3,2,1]) >> ib = loads(dumps(ia))

Traceback (most recent call last):
 . . .
TypeError: can't pickle array objects

[terryjreedy]

Logged In: YES user_id=593130

http://python.org/sf/1281556 appears to be a duplicate. You wish to close it too? (I won't, don't know enough here.)

[2e7e16fe-e205-4c6c-b875-89094129dbaa]

Logged In: YES user_id=1423362

From the comments, it appears that the developers didn't notice that the example used protocol '-1' (highest protocol, which is 2 in Py2.4). Using protocol 0 (default) or 1 will give the TypeError (which is appropriate).

Using protocol 2, pickle.dumps() will accept a pickle object without an exception, but it doesn't seem to pickle it properly (see log below).

Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more
information.
>>> import array
>>> import pickle
>>> a=array.array('B')
>>> s= pickle.dumps(a,2);s
'\x80\x02carray\narray\nq\x00)\x81q\x01.'
>>> pickle.load(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "C:\Python24\lib\pickle.py", line 1390, in load
    return Unpickler(file).load()
  File "C:\Python24\lib\pickle.py", line 855, in __init__
    self.readline = file.readline
AttributeError: 'str' object has no attribute 'readline'
>>>
>>> a=array.array('B', [1,2,3,4,5])
>>> a
array('B', [1, 2, 3, 4, 5])
>>> s2=pickle.dumps(a,2);s2
'\x80\x02carray\narray\nq\x00)\x81q\x01.'

[ef6b2a61-f027-4805-a66a-cde4eee277c3]

Logged In: YES user_id=341410

mbrighton:

You get a string from pickle.dumps(), and to load strings, you should use pickle.loads() .

The AttributeError you are recieving is the result of using pickle.load() with a string, not a file object, as required, and as described by the documentation.

[2e7e16fe-e205-4c6c-b875-89094129dbaa]

Logged In: YES user_id=1423362

Thatks for catching that. Unfortunatly, that's just a function of me editing the example. I ment to edit that out and instead took out the exception I was trying to show. Here's what I ment to demonstrate:

>>> import array
>>> import pickle
>>> a=array.array('B')
>>> s= pickle.dumps(a,2);s
'\x80\x02carray\narray\nq\x00)\x81q\x01.'
>>> pickle.loads(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "C:\Python24\lib\pickle.py", line 1394, in loads
    return Unpickler(file).load()
  File "C:\Python24\lib\pickle.py", line 872, in load
    dispatch[key](self)
  File "C:\Python24\lib\pickle.py", line 1097, in load_newobj
    obj = cls.__new__(cls, *args)
TypeError: array() takes at least 1 argument (0 given)

The part of the problem appears to be with dumps(), as the same string is returned independent of the array provided:

>>> pickle.dumps(array.array('B'),2)
'\x80\x02carray\narray\nq\x00)\x81q\x01.'
>>> pickle.dumps(array.array('B', [1,2,3,4,5]),2)
'\x80\x02carray\narray\nq\x00)\x81q\x01.'
>>> pickle.dumps(array.array('B', [230,]*100),2)
'\x80\x02carray\narray\nq\x00)\x81q\x01.'
>>> pickle.dumps(array.array('H', [1,2,3,4,5]),2)
'\x80\x02carray\narray\nq\x00)\x81q\x01.'