search

tencent-alchemy / Alchemy

https://alchemy.tencent.com/
MIT License
115 stars 38 forks source link

The leaderboard evaluation code might be wrong? #12

Closed mooninrain closed 5 years ago

mooninrain commented 5 years ago

I see this in the submission description.

The data entries should be sorted in an ascending order of GDB ID.

However, when I submit the original valid_target.csv(which is not sorted), I get 0.000; when I submit a sorted one, I get 1.176 MAE.

GuangyongChen commented 5 years ago

The leaderboard evaluation code is correct. The reason for your case is that the valid_target.csv kept on our server is just the one we sent to you and is not stored in the ascending order. Since you have obtained the original valid_target.csv, we do not update the valid dataset on our server. You can just submit your predictions results of the test set following our guides. Thanks!