search

aalto-speech / morfessor

Morfessor is a tool for unsupervised and semi-supervised morphological segmentation
http://morpho.aalto.fi
BSD 2-Clause "Simplified" License
185 stars 29 forks source link

Unpickling a binary model fails #12

Closed thammegowda closed 6 years ago

thammegowda commented 6 years ago

Hi, I am trying to load a model on python3.6 using the python API, but it fails.

Python 3.6.2 |Continuum Analytics, Inc.| (default, Jul 20 2017, 13:51:32)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import morfessor
>>> io = morfessor.MorfessorIO()
>>> mf = 'something.morfmodel.bin'
>>> model = io.read_binary_model_file(mf)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/someone/libs/miniconda3/envs/py3/lib/python3.6/site-packages/morfessor/io.py", line 179, in read_binary_model_file
    model = pickle.load(fobj)
AttributeError: 'ConstrNode' object has no attribute '__dict__'

It is possible that the model may have been trained on the python 2 (i am not sure, my coworker trained it). Questions is, shouldn't the model trained on python 2 work on python 3 (considering the same code is used)?

thammegowda commented 6 years ago

Gotcha! ConstrNode is a named tuple https://github.com/aalto-speech/morfessor/blob/a903366868773981b81875ce9ba2e0111aad92e6/morfessor/baseline.py#L29

problem with namedtuple is that it had a __dict__ attribute in python 2 and then it was removed in python 3 (as per stackoverflow answer)

So, users who migrates code to python 3, but hope to use older models trained on python 2, are in trouble.

thammegowda commented 6 years ago

But... situation is not that bad, I found a work around and raised a PR