generate the dialogue response

[En-J-A]

I tried to generate the dialogue with

python generate_dialogue.py $CHECKPOINT gpt2 $DECODING greedy

I got the result but the generated_response': ['', '', '', '']

when I tried to use the last saved checkpoint as

python generate_dialogue.py --checkpoint="/output/gpt2/checkpoint-350000/pytorch_model.bin"

I got this error

Traceback (most recent call last):
  File "/simpletod/generate_dialogue.py", line 89, in <module>
    tokenizer = GPT2Tokenizer.from_pretrained(model_checkpoint)
  File "/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py", line 1625, in from_pretrained
    f"Calling {cls.__name__}.from_pretrained() with the path to a single file or url is not "
ValueError: Calling GPT2Tokenizer.from_pretrained() with the path to a single file or url is not supported for this tokenizer. Use a model identifier or the path to a directory instead.

how can I generate the response using checkpoint?

[felicitywang]

Hi @AFA-eng, I got the same error of empty response generated. Did you solve this issue? (For the model loading error I think you should use the checkpoint directory path instead of the weights path.)

[En-J-A]

Hi @AFA-eng, I got the same error of empty response generated. Did you solve this issue? (For the model loading error I think you should use the checkpoint directory path instead of the weights path.)

I used the checkpoint directory path but I got the same error: for example :

{'SNG0073.json': {'target_belief': ['taxi destination pizza hut fenditton', 'taxi departure saint johns college', 'taxi leaveat 17:15'], 'target_turn_belief': [['taxi destination pizza hut fenditton', 'taxi departure saint johns college'], ['taxi leaveat 17:15', 'taxi destination pizza hut fenditton', 'taxi departure saint johns college'], ['taxi leaveat 17:15', 'taxi destination pizza hut fenditton', 'taxi departure saint johns college'], ['taxi leaveat 17:15', 'taxi destination pizza hut fenditton', 'taxi departure saint johns college']], 'generated_belief': [], 'generated_turn_belief': [[''], [''], [''], ['']], 'target_response': ['what time do you want to leave and what time do you want to arrive by ?', 'booking completed ! your taxi will be blue honda contact number is 07218068540', 'you are welcome . is there anything else i can help you with today ?', 'you too ! thank you'], 'generated_response': ['', '', '', ''], 'target_action': [['taxi request leave', 'taxi request arrive'], ['taxi inform car', 'taxi inform phone'], ['general reqmore none'], ['general bye none']], 'generated_action': [[], [], [], []], 'target_user': ['i would like a taxi from saint john s college to pizza hut fen ditton .', 'i want to leave after 17:15 .', 'thank you for all the help ! i appreciate it .', 'no , i am all set . have a nice day . bye .'], 'model_context': ['<|endoftext|> <|context|> <|user|> i would like a taxi from saint john s college to pizza hut fen ditton . <|endofcontext|> ', '<|endoftext|> <|context|> <|user|> i would like a taxi from saint john s college to pizza hut fen ditton . <|system|> what time do you want to leave and what time do you want to arrive by ? <|user|> i want to leave after 17:15 . <|endofcontext|> ', '<|endoftext|> <|context|> <|user|> i would like a taxi from saint john s college to pizza hut fen ditton . <|system|> what time do you want to leave and what time do you want to arrive by ? <|user|> i want to leave after 17:15 . <|system|> booking completed ! your taxi will be blue honda contact number is 07218068540 <|user|> thank you for all the help ! i appreciate it . <|endofcontext|> ', '<|endoftext|> <|context|> <|user|> i would like a taxi from saint john s college to pizza hut fen ditton . <|system|> what time do you want to leave and what time do you want to arrive by ? <|user|> i want to leave after 17:15 . <|system|> booking completed ! your taxi will be blue honda contact number is 07218068540 <|user|> thank you for all the help ! i appreciate it . <|system|> you are welcome . is there anything else i can help you with today ? <|user|> no , i am all set . have a nice day . bye . <|endofcontext|> ']}}

[ttthy]

Use the checkpoint directory path instead of the model path

[gnnwiki]

Use the checkpoint directory path instead of the model path

Hi @ttthy . Can you get the result reported in paper when using end2end training without dbsearch?

[ttthy]

No, I got very low scores compared to ones reported in paper.

[En-J-A]

No, I got very low scores compared to ones reported in paper.

Have you got generated responses in the output file??

[En-J-A]

e the checkpoint directory path instead of the

I used the directory path , but there are no generated responses in the output file.

[FedericoCampe8]

Same here. @AFA-eng have you figured out a solution for this?

[En-J-A]

Same here. @AFA-eng have you figured out a solution for this?

no .

[ShuoZhangXJTU]

I found that you can fix the empty generation problem by adding the "<|belief|>" token to the prompt, i.e., text = '{} <|context|> {} <|endofcontext|> <|belief|>'.format(tokenizer._bos_token, tmp_text)