search

rsennrich / subword-nmt

Unsupervised Word Segmentation for Neural Machine Translation and Text Generation
MIT License
2.19k stars 465 forks source link

Facebook ParlAI Blender subword-nmt fails #89

Closed gibmaxn closed 2 years ago

gibmaxn commented 4 years ago

I'm trying to use the new Facebook BlenderBot. This includes ParlAI and also subword-nmt. When I attempt to run the examples I get an error indicating that subword-nmt needs to be installed.
Links to ParlAI:
https://parl.ai/#getstarted https://parl.ai/projects/recipes/ Link to BlenderBot: https://ai.facebook.com/blog/state-of-the-art-open-source-chatbot

The result I'm seeing: ilab@ilab-jupyterhub:~/ParlAI$ pip install subword-nmt Collecting subword-nmt Using cached https://files.pythonhosted.org/packages/74/60/6600a7bc09e7ab38bc53a48a20d8cae49b837f93f5842a41fe513a694912/subword_nmt-0.3.7-py2.py3-none-any.whl Installing collected packages: subword-nmt Successfully installed subword-nmt-0.3.7

ilab@ilab-jupyterhub:~/ParlAI$ sudo python3 parlai/scripts/safe_interactive.py -t blended_skill_talk -mf zoo:blender/blender_90M/model [ warning: overriding opt['task'] to blended_skill_talk (previously: internal:blended_skill_talk,wizard_of_wikipedia,convai2,empathetic_dialogues )] [ warning: overriding opt['model_file'] to /home/ilab/ParlAI/data/models/blender/blender_90M/model (previously: /checkpoint/edinan/20200210/baseline_BST_retnref/lr=7.5e-06_attention-dropout=0.0_relu-dropout=0.0/model )] Dictionary: loading dictionary from /home/ilab/ParlAI/data/models/blender/blender_90M/model.dict [ num words = 54944 ] Traceback (most recent call last): File "parlai/scripts/safe_interactive.py", line 85, in safe_interactive(parser.parse_args(print_args=False), print_parser=parser) File "parlai/scripts/safe_interactive.py", line 59, in safe_interactive agent = create_agent(opt, requireModelExists=True) File "/home/ilab/ParlAI/parlai/core/agents.py", line 406, in create_agent model = create_agent_from_opt_file(opt) File "/home/ilab/ParlAI/parlai/core/agents.py", line 361, in create_agent_from_opt_file return model_class(new_opt) File "/home/ilab/ParlAI/parlai/core/torch_generator_agent.py", line 422, in init super().init(opt, shared) File "/home/ilab/ParlAI/parlai/core/torch_agent.py", line 710, in init self.dict = self.build_dictionary() File "/home/ilab/ParlAI/parlai/core/torch_agent.py", line 793, in build_dictionary d = self.dictionary_class()(self.opt) File "/home/ilab/ParlAI/parlai/core/dict.py", line 303, in init self.bpe = bpe_factory(opt, shared) File "/home/ilab/ParlAI/parlai/utils/bpe.py", line 79, in bpe_factory bpe_helper = SubwordBPEHelper(opt, shared) File "/home/ilab/ParlAI/parlai/utils/bpe.py", line 285, in init "Please run \"pip install 'git+https://github.com/rsennrich" RuntimeError: Please run "pip install 'git+https://github.com/rsennrich/subword-nmt.git#egg=subword-nmt'" ilab@ilab-jupyterhub:~/ParlAI$

gibmaxn commented 4 years ago

I was able to fix this using pip3 install instead of pip. This works: pip3 install subword-nmt This does not work: pip install subword-nmt

rsennrich commented 4 years ago

thanks for reporting this, and finding a solution yourself. The right place to fix this would be to update the error messages in ParlAI to ensure that the Python3 version is being installed - could you open an issue there?