Identify how to distinguish challenge from benchmark?

[tschaffter]

• Look at ChaHub to see if they have a benchmark schema
• Challenge platforms that distinguish between challenges and benchmarks? @vpchung

For our early prototype, we can filter based on whether Challenge.endDate is defined.

[vpchung]

Can we discuss further whether this distinction between challenge and benchmark is needed? imo, a benchmarking challenge is still a challenge?

[vpchung]

That said, I did not see a distinction on Chahub during my perusal of the filters and search box. I did try looking at some specific cards, but it seems like the information displayed are the same, e.g.

Example challenge card

Example benchmark card

[tschaffter]

@vpchung Absolutely, happy to discuss. The information that a prospective participant is probably looking for is "what is the time window during which I can make a submission". In the case of a challenge with an end date, a participant can use this information to make a decision on whether he/she can contribute to the challenge. Maybe the participant will be busy with other things during the challenge and then will decide to no longer track this challenge.

A question I have is whether not specifying an end date for a challenge (benchmark case) is explicit enough for the prospective participant to identify that submission will be accepted for the foreseeable future. Maybe we can solve this by making the client display a message instead of an empty "End date" value.

Another question is whether ROCC users wants to filter challenges by Challenge or Benchmark. To be thorough, here are the different combination:

• Challenge
• Challenge, then becomes a Benchmark
• Benchmark from the start

From a challenge organizer perspective, maybe you don't want to say that your challenge is going to become a benchmark otherwise some participants may think "I will submit later" (i.e. after the challenge) unless there is a strong incentive. However, I challenge organizer may be interested in showing AFTER the completion of the challenge that submissions are still accepted.

I could say more but will save this for a live chat.