Closed faroit closed 5 years ago
Hi faroit
There is no paper. To answer your questions, evaluation metrics are developed in collaboration with challenge organizers. How the computations are carried out also depends on the specific challenge, but yes, we can run participant's code. We use an evaluation infrastructure for that which we haven't open sourced. Private data for scoring is handled also on a case by case basis - every challenge is somewhat different.
Btw, crowdAI is not maintained anymore, we have created AIcrowd as a new entity and are developing there. The frontend will remain open source, and the backend will remain private.
Hi
I'd like to know more about how crowdAI works behind the scenes. I started studying the rails code. However, is there a research paper or presentation of the overall system design?
I am especially interested in