pytries / marisa-trie

Static memory-efficient Trie-like structures for Python based on marisa-trie C++ library.
https://marisa-trie.readthedocs.io/en/latest/
MIT License
1.03k stars 91 forks source link

BytesTries saved with `Trie.save` cannot be loaded with `Trie.load` #37

Closed rspeer closed 7 years ago

rspeer commented 7 years ago

Using marisa-trie 0.7.4, on Python 3.5.1:

>>> t1 = marisa_trie.BytesTrie([('a', b'a'), ('ab', b'b'), ('ac', b'c')])
>>> t1.save('/tmp/t1.marisa')
/home/rspeer/.virtualenvs/lum/bin/ipython:1: DeprecationWarning: Trie.write is deprecated and will be removed in marisa_trie 0.8.0. Please use Trie.save instead.
  #!/home/rspeer/.virtualenvs/lum/bin/python3.5
>>> t2 = marisa_trie.Trie()
>>> t2.load('/tmp/t1.marisa')
/home/rspeer/.virtualenvs/lum/bin/ipython:1: DeprecationWarning: Trie.save is deprecated and will be removed in marisa_trie 0.8.0. Please use Trie.load instead.
  #!/home/rspeer/.virtualenvs/lum/bin/python3.5
<marisa_trie.Trie object at 0x7f4e9ea552d0>
>>> t2.keys()
Traceback (most recent call last):
  File "<ipython-input-23-28b5ae76f2b3>", line 1, in <module>
    t2.keys()
  File "src/marisa_trie.pyx", line 267, in marisa_trie._Trie.keys (src/marisa_trie.cpp:6279)
  File "src/marisa_trie.pyx", line 278, in marisa_trie._Trie.keys (src/marisa_trie.cpp:6172)
  File "src/marisa_trie.pyx", line 403, in marisa_trie._UnicodeKeyedTrie._get_key (src/marisa_trie.cpp:8108)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 2: invalid start byte
rspeer commented 7 years ago

Never mind, this is a duplicate of #18.