Open xin-shinzheng opened 1 month ago
Thanks for catching the error! It should be fixed now.
Hi,
I just realized that the OpenWebText data should not be used for testing. Therefore, your previous implementation on line 46 for i, testDayIdx in enumerate(testDayIdxs)
is indeed correct. In this context, i
should be used as the index for the competition dataset, while testDayIdx
should be used as the index for the test set.
In
neural_seq_decoder/scripts/eval_competition.py
, could you please clarify the data selection for test_loader (lines 45 and 46) for both "competition" and "test"?Should the selection on line 46 apply to both "competition" and "test" data, considering loadedData["competition"] is already being filtered with [4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 16, 18, 19, 20] in formatCompetitionData.ipynb?
Thank you in advance for your help!