search

codalab / codalab-competitions

CodaLab Competitions
https://codalab.lisn.fr
Other
512 stars 128 forks source link

How to display the results from scoring program #3352

Closed YaoZhang93 closed 1 year ago

YaoZhang93 commented 1 year ago

Hi,

I am developing a scoring program for an unpublished competition. Several metrics are calculated, and I want to know how to show the results on the result pages. Here is the result page from the Iris template:

image

Best regards, Yao

Didayolo commented 1 year ago

Hi,

Let's take the Iris challenge as an example.

In the file competition.yaml, the key of the score is defined. In this example, the key of "Prediction score" is set1_score (line 74).

The score output from the scoring program needs to be written in a score.txt file, with the key set1_score. So, after the run, the score.txt file should contain something like set1_score: 0.78. See lines 75 and 114.

I hope this helps.

YaoZhang93 commented 1 year ago

@Didayolo Thanks for your crystal clear instrution. I have already start a competition by uploadling the template competition bundle. My score.txt is like this:

image

And I tried to show "Tumo_DSC", "Tumor_NSD", "Organ_DSC", and "Organ_NSD" in the result page. So I tried to edit the yaml file following your instruction like this:

image

However, the result page remained like this:

image

Did I missing something?

Didayolo commented 1 year ago

Did I missing something?

No, actually it is a known bug that the editor fails to edit leaderboard keys and labels... Sorry for the inconvenience.

You may need to re-upload the competition bundle, with the right keys and labels in the competition.yaml file.

Didayolo commented 1 year ago

(see #2686)

YaoZhang93 commented 1 year ago

OK, I see. Is there any way to download the revised competition bundle? I found a button to download the original bundle but I have updated much information based on the original bundle. It will take too much time if I have to start again from the original bundle.

Didayolo commented 1 year ago

Yes, there is a way.

Click on the grey organizer feature button called "Dump". From this interface, you can create a dump (basically it save the current state of your competition as a new bundle) and then download it.

https://github.com/codalab/codalab-competitions/wiki/User_Running-a-Competition#dumping-a-competition

YaoZhang93 commented 1 year ago

Goit it. Many thanks.