search

PHME-Datachallenge / Data-Challenge-2022

7 stars 7 forks source link

Final ranking #12

Open DaiJunyan opened 2 years ago

DaiJunyan commented 2 years ago

Dear Organizer,

Does the leaderboard count as a final ranking? Will test data be used to calculate scores and announce them before the competition ends?

Best regards, Junyan

dgiordan88 commented 2 years ago

Dear Junyan, The current leaderboard is computed using the Training set. The final ranking will be computed using a separate Test set.

We are considering the possibility to publish the leader board computed by using the Test set. Keeps checking the repository for updates on this manner.

Best

Danilo

sXperfect commented 2 years ago

Dear Organizer,

Assume that in the following days the leaderboard will be evaluated based on the test set. Which result of each team will be taken as the final result? The best result for each task on test data or the result of the models on the last day of the competition?

Best, Yeremia

marty90 commented 2 years ago

The final ranking will be computed uniquely on the test set.

DaiJunyan commented 2 years ago

Dear Organizer,

The current team ranking on the leaderboard is based on the average value of the three f1 scores. Will the final ranking also be based on this average value? During training, Task 2 and 3 have fewer data than Task 1. Is it fair if we apply equal weights to the three f1 scores?

Best regards, Junyan