Addaci
commented
6 years ago Kaggle competitions are based on supervised learning, and require both a training set ofdata for which the target predicted value already has a ground truth outcome available. Kaggle will use this ground truth as an answer key to score the accuracy of participants’ submissions in real time over the course of the competition. For more information see https://www.kaggle.com/host/business and https://www.kaggle.com/host/research.
Background
Some of the best work done on Kaggle has been on research competitions. They present new, unique machine learning datasets and use-cases to the community, and we want to make them available to Kagglers on a more regular basis.
Historically, the process of making research competitions available has been ad-hoc and highly circumstantial. Going forward, we want to make it easier for motivated research, academic, and non-profit groups to host open source challenges on Kaggle.
Opportunity
Kaggle will partner with organizations to host up to 5 pro-bono research competitions a year. We are asking interested organizations to submit a brief proposal for Kaggle’s consideration.
Requirements
To be eligible to host a research competition, the host organization must:
- Be an academic, research, or non-profit organization
- Have legal authorization to release the competition dataset publicly
- Post a cash prize
- Offer the prize on condition of winning solutions being open sourced
Selection Criteria
Kaggle will select research hosts on a rolling basis over the entire year. We will opportunistically reach out to prospective hosts based on the content of their application forms. Primarily, we will be looking for:
Fit - Is the project a cleanly structured supervised learning problem? Is there ample data available to support a successful competition?
Feasibility - Will it be logistically possible to make the data available for the competition? Is the hosting organization in a position to facilitate a successful competition?
Impact - What is the scope of the problem this competition addresses? How impactful would the open source solutions be on the industry this competition supports?
Kaggle and the prospective host will go through a vetting process before setting up and launching a competition. This vetting process will include a review of the dataset, some premodeling of the problem, and selection of an evaluation metric. The host organization will also need to sign a data-use agreement and be ready to transfer Kaggle the full value of the prize pool upon launch of the competition. Kaggle will facilitate the payment of prizes to winners at the competition close.
@gbohner has suggested that the Signs of Literacy community set up a Kaggle research competition to perform a Proof of Concept on preliminary datasets of markes, initials & signatures. Gergo states that "The Kaggle community is adept at applying a range of potential algorithms quickly to solve similar problems, and it will give you a good idea of 1) how well AI can replace the volunteers on your task(s), and a starting point for 2) what are the kind of algorithms you might want to improve upon in a collaboration with a research lab."