Feature Request: Display String Values in a Column on the Leaderboard

[wonjininfo]

Hello,

Thank you for creating and open-sourcing this valuable project!

I am in the process of posting a benchmark dataset from an academic paper and encountered an obstacle with the leaderboard UI. While uploading baseline results reported in the paper, I noticed there isn't an option to display model names directly on the leaderboard. I believe other researchers who want to submit to our benchmark would also want to submit with various settings like using various LLMs or ensemble approaches and mark them with their model name or settings.

An example of this will be adding a column for model names, similar to the format used on the SQuAD dataset webpage: https://rajpurkar.github.io/SQuAD-explorer/.

Currently, my workaround involves creating a Model name column and reading this attribute from a file to post it as a string on the leaderboard.

However, the current system settings do not support displaying string values on the leaderboard. Although my scoring program can read the "modelname" file, it's unable to present this information on the leaderboard and only showing n/a.

This is current shape of my competition.

Do you have plans to add a feature that allows displaying string values on the leaderboard? I am considering creating a pull request to implement this, but I would appreciate any suggestions or guidance you could provide to help me get started.

• A link to your competition.: https://www.codabench.org/competitions/2064/#/results-tab

Best regards, WonJin

[ihsaan-ullah]

Hi @wonjininfo

You can easily add a column in the leaderboard by using Factsheet/Metadata. You can find details here: https://github.com/codalab/codabench/wiki/Yaml-Structure#fact-sheet

[wonjininfo]

Hi @ihsaan-ullah , Thank you so much for your reply. It works well and applied to our benchmark leaderboard!