Cannot reproduce BLEU score on test set

IBM / Graph2Seq

Graph2Seq is a simple code for building a graph-encoder and sequence-decoder for NLP and other AI/ML/DL tasks.

Apache License 2.0

237 stars 61 forks source link

Cannot reproduce BLEU score on test set #8

Open Hazoom opened 3 years ago

Hazoom commented 3 years ago

Hi,

I couldn't find a way in the code to reproduce the BLEU score of 38.97 you published in your paper on the test set of WikiSQL.

Can you please instruct me on how to reproduce it?

Thanks.

khieu commented 3 years ago

I have the same question. The format of WikiSQL seems to be different from the input format in the data/no_cycle folder. Any suggestion on creating the input for different dataset?

teddylfwu commented 3 years ago

Dear All,

Please see our newly released graph4nlp library: https://github.com/graph4ai/graph4nlp, which have implemented many GNN methods like Graph2Seq and Graph2Tree models.