search

huggingface / transfer-learning-conv-ai

🦄 State-of-the-Art Conversational AI with Transfer Learning
MIT License
1.73k stars 431 forks source link

using question generation in isolation README.md clarification #55

Closed mooreniemi closed 4 years ago

mooreniemi commented 4 years ago

I was curious to try using the question generation component. Based on the overall README, I might expect to be able to run interact.py without any arguments, but this doesn't work:

ERROR:pytorch_pretrained_bert.modeling_openai:Model name '' was not found in model name list (openai-gpt). We assumed '' was a path or url but couldn't find files pytorch_model.bin and config.json at this path or url.
Traceback (most recent call last):
  File "question-generation/interact.py", line 238, in <module>
    run()
  File "question-generation/interact.py", line 149, in run
    model.to(args.device)
AttributeError: 'NoneType' object has no attribute 'to'

That is ok, I downloaded a pretrained model myself from Google drive and tried interact.py again (python3 question-generation/interact.py --model_checkpoint ~/Downloads/gpt2_corefs_question_generation/). When I did so, I first hit the below issue:

  File "question-generation/interact.py", line 238, in <module>
    run()
  File "question-generation/interact.py", line 152, in run
    data = get_positional_dataset_from_file(tokenizer, args.filename)
  File "/Users/amooren/Code/squash-generation/question-generation/dataloader.py", line 81, in get_positional_dataset_from_file
    with open(file, 'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'data/instances_dev.pkl'

Reading the overall README, I inferred that I needed to download instances_dev.pkl, and I found something closely named instances_dev.pickle and instances_corefdev.pickle here. I assumed I'd need the coref labeled one, as the pretrained folder is named "gpt2corefs_question_generation". But when I tried to use this, I hit the next issue:

Traceback (most recent call last):
  File "question-generation/interact.py", line 238, in <module>
    run()
  File "question-generation/interact.py", line 168, in run
    para_index = inst["para_index"]
KeyError: 'para_index'

Would you mind clarifying the steps to test out just this component? It would greatly speed up my efforts.

sshleifer commented 4 years ago

It looks to me like you have other code (e.g. dataloader.py) from another repository. We don't have any instances_dev.pkl or dataloader.py

mooreniemi commented 4 years ago

Oh no, my apologies! I opened this in the wrong repo, as squash links to this one. I'll close this here.

Vivaswan13 commented 4 years ago

@mooreniemi I am also facing the same problem, did you find the solution?