Missing package dependency for hallucination project

rerngvit commented 3 years ago

Bug description I was trying to test the recently released retrieval augmented generation model in ParlAI 1.3, e.g., just trying to run $parlai interactive -mf zoo:hallucination/bart_rag_token/model. However, it seems that the some package dependency for the project have not been added to requirements.txt. This would result in an error shown below.

Reproduction steps

Create a separate pip venv
Git clone latest: git clone https://github.com/facebookresearch/ParlAI
Install required packaged: pip install -r requirement.txt
parlai interactive -mf zoo:hallucination/bart_rag_token/model

Expected behavior The interactive chatting with the pretrained model should be started to run

Logs

Traceback (most recent call last):
  File "/home/jupyter/venvs/dev_parlai/bin/parlai", line 11, in <module>
    load_entry_point('parlai', 'console_scripts', 'parlai')()
  File "/home/jupyter/src/workspace/ParlAI/parlai/__main__.py", line 14, in main
    superscript_main()
  File "/home/jupyter/src/workspace/ParlAI/parlai/core/script.py", line 316, in superscript_main
    opt = parser.parse_args(args)
  File "/home/jupyter/src/workspace/ParlAI/parlai/core/params.py", line 1163, in parse_args
    self.add_extra_args(args)
  File "/home/jupyter/src/workspace/ParlAI/parlai/core/script.py", line 175, in add_extra_args
    sa.choices[args[0]].add_extra_args(args)
  File "/home/jupyter/src/workspace/ParlAI/parlai/core/params.py", line 948, in add_extra_args
    self.add_model_subargs(model, partial)
  File "/home/jupyter/src/workspace/ParlAI/parlai/core/params.py", line 822, in add_model_subargs
    agent = load_agent_module(model)
  File "/home/jupyter/src/workspace/ParlAI/parlai/core/loader.py", line 188, in load_agent_module
    my_module = importlib.import_module(module_name)
  File "/opt/conda/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/jupyter/src/workspace/ParlAI/parlai/agents/rag/rag.py", line 22, in <module>
    from parlai.agents.hugging_face.t5 import T5Agent
  File "/home/jupyter/src/workspace/ParlAI/parlai/agents/hugging_face/t5.py", line 15, in <module>
    from transformers import T5ForConditionalGeneration
ModuleNotFoundError: No module named 'transformers'

Additional context Note that it seems a specific pinned version is required, since just pip install transformers will result in a conflict with another required library. Also, from what mentioned in the paper, a package for faiss will likely be needed as well.

klshuster commented 3 years ago

Hi there --> yes, you'll need to pip install transformers==4.3.3 to use the hallucination project models.

Additionally, you'll need to run pip install faiss-gpu==1.7.0 (or faiss-cpu, either works)

We will eventually add each to the requirements in future releases

klshuster commented 3 years ago

as the failure is more informative (and provides instructions for installing), I'll be closing this issue. Please reopen if you have additional questions/concerns

facebookresearch / ParlAI

Missing package dependency for hallucination project #3774