CLEF Challenges

[scarroll32]

This modification provides support for the 'CLEF' challenges model, eg: ImageCLEF.

Specification document

The key tasks required for crowdAI are:

Organizer / Challenge

• [x] Checkbox on the organizer to identify a 'CLEF' organizer
• [x] Template for 'CLEF' to control which challenge fields are visible / required (This feature will be useful but adds a lot of complexity - will be addressed at some future point, with conditional view logic based on the CLEF organizer box for now).
• [x] Selector on datasets page to choose the datasets from another CLEF challenge
• [ ] Ability for an organizer account to invite other participants or remove a participant from the organization
• [x] Addition of a 'Participants' tab to the challenge #358
• [x] Status and approval buttons on participant's tab (submitted, approved, rejected) (note that the rejected flow has not been discussed yet)

Participants

• [ ] Addition of Team Name + team leader https://github.com/crowdAI/crowdai/issues/325
• [x] Addition of Affiliation field https://github.com/crowdAI/crowdai/issues/353
• [x] Addition of address + country fields (visible to admins only, optional for non-CLEF) https://github.com/crowdAI/crowdai/issues/353

Datasets and EUA

• [x] Additional of field on challenge to upload a PDF EAU document https://github.com/crowdAI/crowdai/issues/357
• [x] Update datasets page for CLEF challenges to present the EAU document (with instructions)
• [x] Upload document button
• [ ] Page changes to pending and the regular datasets pages upon submission and approval of the document
• [ ] Possibly add notification emails (or else add only to Notifications https://github.com/crowdAI/crowdai/issues/306 )

Submissions

To be further developed, but in general the page will make an API call to the broker, which wraps tje client, and then sends the file or the S3 key of the file.

[ieggel]

Status and approval buttons on participant's tab (submitted, approved, rejected)

Could we also have the downloaded status in order to mark the participant as registered? Or maybe you have a better idea? It is important for us that the registration is open before the actual data release, so there should be a definition of when somebody is considered as registered. Or do we want to introduce the terms&licenses which the user has to accept before the EUA page appears? In this case we could insert a sentence like "By accepting these terms you will also be registered to this challenge" or something similar? What do you think?

[scarroll32]

We could use these statuses:

• requested - user has downloaded the form but but not submitted it
• submitted - user has uploaded the signed form
• registered - ImageCLEF admin has approved the user and they start accessing the data as soon as it is released
• rejected - something is wrong with the form. I think the best action here is the ImageCLEF admin emails the user to discuss.

[ieggel]

Ok. This seems reasonable. So let's stick with these statuses. An ImageClef participant will be considered as registered as soon as his signed EUA has been validated.

[scarroll32]

@ieggel The Tasks tab will have the EAU and also the dataset files.

Will this be visible to any logged in user or only to the organizer?

[ieggel]

@seanfcarroll

I think it's better if this tab is only visible to the organizer. The participants can see the EUA and the dataset files on the challenge page.

[scarroll32]

Thanks @ieggel

[scarroll32]

Implemented