I would like to make sure I'm using correct dataset handling to obtain the results in Table 3 of the paper. Would you please help clarify the following questions?
Based on my understanding of the paper, Table 3 reports test set results over all datasets.
For Advising, Geo, Scholar and ATIS the datasets were split into train/dev/test. When testing the model on test set, shall we train the models using only the train set or the train + dev set combined?
For Academic, IMDB, Restaurants, Yelp there are only cross validation split in the data release. Hence are the numbers over these dataset in Table 3 cross-validation numbers? If not, how are the test set defined for these datasets?
I would like to make sure I'm using correct dataset handling to obtain the results in Table 3 of the paper. Would you please help clarify the following questions?
Based on my understanding of the paper, Table 3 reports test set results over all datasets.
For
Advising
,Geo
,Scholar
andATIS
the datasets were split into train/dev/test. When testing the model on test set, shall we train the models using only the train set or the train + dev set combined?For
Academic
,IMDB
,Restaurants
,Yelp
there are only cross validation split in the data release. Hence are the numbers over these dataset in Table 3 cross-validation numbers? If not, how are the test set defined for these datasets?Thanks for your attention.