Inner defined class do not support pickle (EnumItem)

[duchesnay]

Pickling LimitedDict raise Error, the reason is that EnumItem is an inner class.

Example: from parsimony.utils import LimitedDict, Info import pickle info = LimitedDict(Info.num_iter) o = open("/tmp/toto.pkl", "w") pickle.dump(info, o)

Raise an error:

PicklingError Traceback (most recent call last) ... PicklingError: Can't pickle <class 'parsimony.utils.utils.EnumItem'>: it's not found as parsimony.utils.utils.EnumItem

I suggest to move EnumItem as an outer class

[duboism]

I don't know if that will help but Enums were added in python 3.4 and backported to 2.7 (see https://pypi.python.org/pypi/enum34/ for an implementation by the author of the 3.4 implementation). These classes must be pickable.

[tomlof]

Great find, @duchesnay! I will update.

@duboism, yes, it is available in Python 3.4, but not in 2.7, as you said. Do you know if it will be included in any official updates of Python 2.7.x in the future? As far as I understand, they only include new stuff in the Python 3 lineage.

Thus, we can choose to include a dependency on the enum package from pypi, but I'm not sure it is such a good idea to add dependencies.

Our long-term goal should be to migrate to Python 3, so maybe we can live with a not-so-perfect implementation of an enum class until we migrate (when numpy and scipy work in Python 3)? Or, depending on the license of the enum class from pypi, we could include it entirely our package.

[duboism]

The main advantage of using the enum34 package with python 2.7 is that the API is the same than in 3.4. So this might help the transition to python 3.

[tomlof]

You are right @duboism, that would be the best, and I kind of tried to make it similar, but there are many, many things that the Enum34 class does that this one does not.

The Enum34 package has a BSD license. It does not say which, but, it seems to me that we could ship the enum class with our library. Is that overkill? Should we do it?

[duboism]

Wouldn't it better to list it as dependency for python < 3.4? I have downloaded the source and it's necessary to install it (it's not directly a python file) so distributing it with parsimony maybe tricky.

I think it's not in Ubuntu 12.04 packages (there are similar ones but maybe the API is slightly different) but it is very simple to install and it will be in 14.04 (next LTS which should be available soon (in 2 days apparently)).

[tomlof]

Yes, I think you are right. Though, I am reluctant to add dependencies, and especially so for something that is rarely used in the code. Though, if we have it, we can and probably will use Enums more!

Is pip installed by default with Python on Ubuntu, or would the user have to install that as well?

A trivial alternative would be to do something like:

class Info(object): t = 1 f = 2 num_iter = 4 ...

And thus just use Info.t, though that would yield Into.t == 1 to be true, which is slightly problematic.

Maybe we should wait until 14.04 is released and deployed on NeuroSpin and see if it is available per default in 14.04?

[tomlof]

Oh, I just saw that numpy and scipy is available with Python 3.2 and upwards: http://www.scipy.org/scipylib/faq.html#do-numpy-and-scipy-support-python-3-x

It didn't use to be... I'm getting old ;-)

Maybe we should broaden the discussion and talk about migrating to Python 3.2+ directly?

Or, we could spend some time to make the code work seamlessly in both Python 2.7.x and 3.2+. I don't think we use that many constructs that require a particular version. From the top of my head I can think of two things that are very different between versions: range vs. xrange (lazy evaluation or not) and iterating over dictionaries. But we can make that work between versions with some small effort.

[duboism]

Yeah I also discovered recently that scipy is available for python 3.2.

There was a talk on the transition to python 3 in the recent pycon conference. I can find it but I think they recommended to have the same source compatible with python 2 and python 3 (eventually including run-time checks).

[tomlof]

This is fixed. We should open another issue that discusses the transition to Python 3.2+. I'll close this for now.