search

aalto-speech / morfessor

Morfessor is a tool for unsupervised and semi-supervised morphological segmentation
http://morpho.aalto.fi
BSD 2-Clause "Simplified" License
180 stars 27 forks source link

Fixed #12: unpickling a py2 trained model on py3 #13

Closed thammegowda closed 6 years ago

thammegowda commented 6 years ago

Because the pytho2's collections.namedtuple doesn't unpickle with python 3's collection.namedtuple, I made a work around by defining a class.

it solves the issue I mentioned and closes #12

psmit commented 6 years ago

Hi @thammegowda , thanks for your interest in making Morfessor better. I'm afraid that your solution will have quite an impact on the memory usage of Morfessor.

We have always known (and hopefully told in the documentation) that the binary model format can only be used with the same version of Python. For a model that is compatible with other Python versions, we suggest you use the 'old' text-based model format. For a long time we have planned to make a newer on-disk format that works across Python versions and supports all features, but unfortunately, we have never had the time to finish that.

thammegowda commented 6 years ago

@psmit I didn't know that there is such a difference between namedtuple and the user defined class in python! I will appreciate if you can share any reference about this topic (i like to write efficient code too, so good learning for me)

I am able to use my models with the patch and in my case a little bump in memory was not a blocker. We can close this issue since you mentioned it is a known issue.