search

cffan / neural_seq_decoder

23 stars 13 forks source link

Indexing error in `neural_seq_decoder/scripts /eval_competition.py` #2

Open xin-shinzheng opened 1 month ago

xin-shinzheng commented 1 month ago

In neural_seq_decoder/scripts/eval_competition.py, could you please clarify the data selection for test_loader (lines 45 and 46) for both "competition" and "test"?

Should the selection on line 46 apply to both "competition" and "test" data, considering loadedData["competition"] is already being filtered with [4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 16, 18, 19, 20] in formatCompetitionData.ipynb?

Thank you in advance for your help!

cffan commented 1 month ago

Thanks for catching the error! It should be fixed now.

xin-shinzheng commented 2 weeks ago

Hi,

I just realized that the OpenWebText data should not be used for testing. Therefore, your previous implementation on line 46 for i, testDayIdx in enumerate(testDayIdxs) is indeed correct. In this context, i should be used as the index for the competition dataset, while testDayIdx should be used as the index for the test set.