commonsense / conceptnet5

Code for building ConceptNet from raw data.
Other
2.78k stars 355 forks source link

msgpack related error on database loading #287

Open rominf opened 4 years ago

rominf commented 4 years ago

I'm following instructions on how to build ConceptNet 5.7 (from master branch), I use puppet on Ubuntu 16.04 LTS + Python 3.7, installed using pyenv. I'm getting the following error:

Traceback (most recent call last):
  File "/home/conceptnet/.pyenv/versions/conceptnet5/bin/cn5-read", line 11, in <module>
    load_entry_point('ConceptNet', 'console_scripts', 'cn5-read')()
  File "/home/conceptnet/.pyenv/versions/3.7.7/envs/conceptnet5/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/conceptnet/.pyenv/versions/3.7.7/envs/conceptnet5/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/conceptnet/.pyenv/versions/3.7.7/envs/conceptnet5/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/conceptnet/.pyenv/versions/3.7.7/envs/conceptnet5/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/conceptnet/.pyenv/versions/3.7.7/envs/conceptnet5/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/conceptnet/conceptnet5/conceptnet5/readers/cli.py", line 185, in run_emoji
    emoji.handle_file(input, output)
  File "/home/conceptnet/conceptnet5/conceptnet5/readers/emoji.py", line 27, in handle_file
    out = MsgpackStreamWriter(output_file)
  File "/home/conceptnet/conceptnet5/conceptnet5/formats/msgpack_stream.py", line 14, in __init__
    self.packer = msgpack.Packer(encoding='utf-8')
  File "msgpack/_packer.pyx", line 118, in msgpack._cmsgpack.Packer.__init__
TypeError: __init__() got an unexpected keyword argument 'encoding'
Traceback (most recent call last):
  File "/home/conceptnet/.pyenv/versions/conceptnet5/bin/cn5-read", line 11, in <module>
    load_entry_point('ConceptNet', 'console_scripts', 'cn5-read')()
  File "/home/conceptnet/.pyenv/versions/3.7.7/envs/conceptnet5/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/conceptnet/.pyenv/versions/3.7.7/envs/conceptnet5/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/conceptnet/.pyenv/versions/3.7.7/envs/conceptnet5/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/conceptnet/.pyenv/versions/3.7.7/envs/conceptnet5/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/conceptnet/.pyenv/versions/3.7.7/envs/conceptnet5/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/conceptnet/conceptnet5/conceptnet5/readers/cli.py", line 185, in run_emoji
    emoji.handle_file(input, output)
  File "/home/conceptnet/conceptnet5/conceptnet5/readers/emoji.py", line 27, in handle_file
    out = MsgpackStreamWriter(output_file)
  File "/home/conceptnet/conceptnet5/conceptnet5/formats/msgpack_stream.py", line 14, in __init__
    self.packer = msgpack.Packer(encoding='utf-8')
  File "msgpack/_packer.pyx", line 118, in msgpack._cmsgpack.Packer.__init__
TypeError: __init__() got an unexpected keyword argument 'encoding'
[Sun May 10 05:24:25 2020]
Error in rule read_emoji:
    jobid: 276
    output: data/edges/emoji/my.msgpack
    shell:
        cn5-read emoji data/raw/emoji/my.xml data/edges/emoji/my.msgpack
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Removing output files of failed job read_emoji since they might be corrupted:
data/edges/emoji/my.msgpack
[Sun May 10 05:24:25 2020]
Error in rule read_emoji:
    jobid: 274
    output: data/edges/emoji/mr.msgpack
    shell:
        cn5-read emoji data/raw/emoji/mr.xml data/edges/emoji/mr.msgpack
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

I solved it by removing encoding argument everywhere in conceptnet5/formats/msgpack_stream.py. I'm not sure whether it's the right approach.