Suggestions for V2/V1.5 Wishlist

[madclam]

I am referencing here old suggestions that we can revisit for the design of V2:

• Hall of fame: https://github.com/codalab/codalab-competitions/issues/1196
• Calendar view of events (Chagle feature?) https://github.com/codalab/codalab-competitions/issues/1887
• Enhanced user profile info: https://github.com/codalab/codalab-competitions/issues/511
• Better admin features: https://github.com/codalab/codalab-competitions/issues/1025, https://github.com/codalab/codalab-competitions/issues/1617
• Execution wrapper: https://github.com/codalab/codalab-competitions/issues/991, related to https://github.com/codalab/codalab-competitions/issues/1913
• Dataset submission: https://github.com/codalab/codalab-competitions/issues/1310
• Multiple tracks: https://github.com/codalab/codalab-competitions/issues/1201
• Improve user panel: https://github.com/codalab/codalab-competitions/issues/1245
• Improve leaderboard: https://github.com/codalab/codalab-competitions/issues/1341; change leaderboard from editor: https://github.com/codalab/codalab-competitions/issues/1555; mode to show multiple submissions by the same user/team https://github.com/codalab/codalab-competitions/issues/1614
• re-think coopetitions https://github.com/codalab/codalab-competitions/issues/1618
• create evaluation with dependencies #1948
• automatic tweeting to get presence on twitter #1192
• nicely display "time remaining" in hours/seconds underneath competition end date
• More meaningful error messages https://github.com/codalab/codalab-competitions/issues/1626
• Better email notifications https://github.com/codalab/codalab-competitions/issues/1637
• Better sorting of competitions into student/research/trial/ etc. avoid accumulating dummy competitions https://github.com/codalab/codalab-competitions/issues/1641
• Better exception handling: https://github.com/codalab/codalab-competitions/issues/1647
• Pre-registering groups of people: https://github.com/codalab/codalab-competitions/issues/1649
• Elastic compute workers: https://github.com/codalab/codalab-competitions/issues/1606, https://github.com/codalab/codalab-competitions/issues/1889
• Allow NOT using cloud storage: https://github.com/codalab/codalab-competitions/issues/1720
• Work on task program and ingestion program: https://github.com/codalab/codalab-competitions/issues/1906, https://github.com/codalab/codalab-competitions/issues/1913
• Clean up the notion of team: https://github.com/codalab/codalab-competitions/issues/1648
• Completely close accounts: https://github.com/codalab/codalab-competitions/issues/2006, also opt-out of notifications https://github.com/codalab/codalab-competitions/issues/2005
• Changes to ingestion program https://github.com/codalab/codalab-competitions/issues/1987

[ckcollab]

• Points system/gamification #909
• Submit via worksheet
• Ability to "manage submissions" -- Instead of phase-to-phase migrations, create a nice screen for handling running submissions. You can move them between phases manually. You can lock phases down, open them, etc. easily.
• Image optimization after any image has been uploaded (resize properly, tinypng, compression, etc.)
• Add a "slug" section when you edit competitions. The "slug" will be the competition name in the url. https://codalab.org/c/<your slug> will redirect to your competition

[madclam]

This is a list of bugs or enhancements with lower priority:

• [x] Detailed results not properly displayed: https://github.com/codalab/codalab-competitions/issues/1741
• Add search bar when logged in: https://github.com/codalab/codalab-competitions/issues/1740
• Make it possible to download multiple submissions: https://github.com/codalab/codalab-competitions/issues/1632
• Fix static files in competition dumps: https://github.com/codalab/codalab-competitions/issues/1897
• Leaderboard does not behave when phases are added from the editors: https://github.com/codalab/codalab-competitions/issues/1693
• Show ranks in separate fields in CSV files: https://github.com/codalab/codalab-competitions/issues/1672
• Improve mail message capabilities (include HTML, attachments, etc.) https://github.com/codalab/codalab-competitions/issues/1654
• Improve HTML editor https://github.com/codalab/codalab-competitions/issues/1635

[skrish13]

Adding a couple of features from #2115

• Filtering based on a very simple, generic tag. Ex: NLP, CV, BioMedical etc
• Sorting wrt End Date.
• Filter for open/closed competitions.

[ckcollab]

• Make it so scoring is not delayed by other submissions, scoring tasks should be higher priority

[Tthomas63]

• Compute worker chunked upload -- replace put_blob
• Make deleting a submission also cancel the Celery task
• Add a "display name" profile field
  • Similar to organizer deleting submission cancels it, we should have a "cancel submission" button that regular user can run
  • Create a bundle checker
  • migrate users to new competition
  • Add a simple button "Request Chahub Access"

[Didayolo]

Submissions limit

• Less than 1 maximum_submissions_per_day: #2434
• Set a maximum number of submissions per team? #2751

[Didayolo]

Adding more information in a string type column in the leaderboard:

• Show method details on leaderboard: #2378
• How to add string column in leaderboard table: #2404
• Can I add text type column for leaderboard: #2406

[Didayolo]

Registrations and un-registrations

• Spam messages in Forum: #2627
• Hall of fame opt in + explanation: #2349
• [Suggestion] Registration Timestamp: #2203

[Didayolo]

GUI

• [x] Add "health/simple_status" page to "Admin monitoring links"

Baselines

• [ ] Being able to to mark some submissions as baselines in the leaderboard (highlight)
• [x] Let the organizers put several baseline scores in the leaderboard without creating new accounts https://groups.google.com/g/codalab-competitions/c/gFtfFoaBlRs

Example ticket #3155

Documentation

• Improve documentation about competition Docker image and how to define the evaluation environment https://groups.google.com/g/codalab-competitions/c/-9nkgisJfAQ