GPT-2.py output a json with low bleu3

wenhuchen / LogicNLG

The data and code for ACL2020 paper "Logical Natural Language Generation from Open-Domain Tables"

MIT License

167 stars 22 forks source link

GPT-2.py output a json with low bleu3 #1

Closed whuFSN closed 4 years ago

whuFSN commented 4 years ago

Hello For GPT-2.py model, I decode the sentences with the script CUDA_VISIBLE_DEVICES=0 python GPT2.py --do_test --load_from models/GPT_ep8.pt but I got the outputs GPT_gpt2_0.03.json which has a low bleu_3, the content in the GPT_gpt2_0.03.json like this:

"2-18424778-6.html.csv": [ "", "", "", "", "" ], "2-1458666-4.html.csv": [ "", "", "", "wins iaa mwcwcwcwcwcwcwcwcwcwcw", "" ]

Does it because of args.model==gpt2? Or it should be gpt2-medium

wenhuchen commented 4 years ago

Hi there, I couldn't reproduce your error. Could you use "pytorch=1.4" and "transformers=2.6", and make sure you unzip all_csv.zip. From the output, it seems the model is not properly loaded. Could you make sure the model.load_state_dict is actually working?

whuFSN commented 4 years ago

Thank you for your answers. I update pytorch to 1.4 and transformers to 2.6. It solved my problem.