search

ethz-spylab / satml-llm-ctf

Code used to run the platform for the LLM CTF colocated with SaTML 2024
https://ctf.spylab.ai
MIT License
25 stars 6 forks source link

Defenses Leaderboard tiebreaker #62

Closed adrgs closed 9 months ago

adrgs commented 9 months ago

Could the leaderboard be updated such that it shows the correct order of the defenses given equal Value of Defense score?

Would help attacking teams determine on which defenses to focus next.

dpaleka commented 9 months ago

We can do the "sum of attacker scores" tiebreaker soon. The timestamp tiebreaker is a bit too cumbersome in the leaderboard, sorry. I hope it won't be relevant.

dpaleka commented 9 months ago

I guess this is done. We can produce a list of when defenses were first broken once every defense is broken and put it on the website; would this be a satisfactory solution?

adrgs commented 9 months ago

Yes, that would be good. Thanks for the update!