search

huggingface / transfer-learning-conv-ai

🦄 State-of-the-Art Conversational AI with Transfer Learning
MIT License
1.73k stars 431 forks source link

interact.py killed on second run #44

Closed alelasantillan closed 4 years ago

alelasantillan commented 4 years ago

I did: git clone https://github.com/huggingface/transfer-learning-conv-ai cd transfer-learning-conv-ai pip install -r requirements.txt python -m spacy download en

Then It run once as follows: python3 interact.py

It downloaded the model and I could chat with the bot. I stop the execution with Ctrl+C and then I did again: python3 interact.py

to get: INFO:pytorch_transformers.modeling_utils:loading weights file /tmp/tmpzwe_u3oz/pytorch_model.bin INFO:pytorch_transformers.modeling_utils:Weights from pretrained model not used in OpenAIGPTLMHeadModel: ['multiple_choice_head.summary.weight', 'multiple_choice_head.summary.bias'] INFO:pytorch_transformers.tokenization_utils:Assigning to the bos_token key of the tokenizer INFO:pytorch_transformers.tokenization_utils:Assigning to the eos_token key of the tokenizer INFO:pytorch_transformers.tokenization_utils:Assigning to the pad_token key of the tokenizer INFO:pytorch_transformers.tokenization_utils:Assigning ('', '') to the additional_special_tokens key of the tokenizer INFO:interact.py:Sample a personality INFO:/home/jovyan/work/alela/dan_chatbots/transfer-learning-conv-ai/utils.py:Load tokenized dataset from cache at ./dataset_cache_OpenAIGPTTokenizer Killed

And every time I run pyhton3 interact.py, the same happens. Always Killed.

Somebody can explain?

On the other hand I have no way to exit the program while into the dialogue but to do Ctrl+C

Thank you, Alejandro

sshleifer commented 4 years ago

Hi. Unfortunately I can't replicate this on my machine. Can you try torch.load('dataset_cache_OpenAIGPTTokenizer') in a python shell and see if that works? Alternatively, what version of pytorch/pytorch-transformers are you on?

alelasantillan commented 4 years ago

I was able to avoid the Killed prompt and now the python ./interact.py works well by deleting dataset_cache_OpenAIGPTTokenizer from the local dir before running the command.

But if I Ctrl+C and then I run it again without erasing the file dataset_cache_OpenAIGPTTokenizer it fails. I've tried a lot of usual commands into the dialogue prompt to exit clean, without any success. Using Ctrl+Z works, but because the process is not canceled but paused (it continues after sending the fg command, as usually happens with any excution in linux).

Regarding your question: pytorch-ignite==0.2.1 pytorch-transformers==1.2.0 torch==1.3.0 and torch.load('dataset_cache_OpenAIGPTTokenizer') loads perfectly well within a python3 shell.

Any idea how to finish the dialogue smoothly?

sshleifer commented 4 years ago

Interesting. My versions are the same and I can't replicate. This question might be relevant: https://stackoverflow.com/questions/726690/what-killed-my-process-and-why, specifically seeing whether dmesg -T| grep -E -i -B100 'killed process' says anything informative.

alelasantillan commented 4 years ago

Thank you sshleifer!!! maybe that's the problem. This is a docker running on a mac. There may not be enough resources assigned.