🛠️ grand-challenge-forge

A utility that generates distributable items that help challenge organizers set up their challenge more easily on Grand-Challenge.org.

Install

Install via PyPi:

pip install grand-challenge-forge
grand-challenge-forge --help

📦 Challenge packs

A challenge pack consists of challenge-tailored examples for the following:

A script to automate uploading data to an archive
A submission algorithm that can be submitted to a challenge phase
An evaluation method that evaluates algorithm submissions and generates performance metrics for ranking

Usage

grand-challenge-forge pack-context.json

Will use the context found in pack-context.json and generate a pack at the current working directory in a directory dist/ (default).

Example of the content of pack-context.json

```JSON { "challenge": { "slug": "challenge-slug", "phases": [ { "slug": "phase-slug", "archive": { "url": "https://grand-challenge.org/archives/archive-slug/" }, "algorithm_inputs": [ { "slug": "input-ci-slug", "kind": "Segmentation", "super_kind": "Image", "relative_path": "images/input-value" }, { "slug": "another-input-ci-slug", "kind": "Anything", "super_kind": "File", "relative_path": "another-input-value.json" } ], "algorithm_outputs": [ { "slug": "output-ci-slug", "kind": "Image", "super_kind": "Image", "relative_path": "images/output-value" }, { "slug": "another-output-ci-slug", "kind": "Anything", "super_kind": "File", "relative_path": "output-value.json" } ] }, { "slug": "another-phase-slug", "archive": { "url": "https://grand-challenge.org/archives/another-archive-slug/" }, "algorithm_inputs": [ { "slug": "input-ci-slug", "kind": "Image", "super_kind": "Image", "relative_path": "images/input-value" } ], "algorithm_outputs": [ { "slug": "another-output-ci-slug", "kind": "Anything", "super_kind": "File", "relative_path": "output-value.json" } ] } ] } } ```

Alternatively, you generate a pack by providing a JSON string directly:

grand-challenge-forge --output-dir /tmp '{ "challenge": { "slug": "a-slug"...'

This will output a pack directory in the /tmp directory.

[!NOTE] By default, the forge does quality checks on the pack that may require docker. You can disable these via -n

🏗️ Development

Install locally

Install grand-challenge-forge locally (requires poetry):

git clone https://github.com/DIAGNijmegen/rse-grand-challenge-forge.git
cd rse-grand-challenge-forge
poetry install
poetry run grand-challenge-forge --help

Pre-commit hooks

Several linters and stylers run to check the formatting during continuous integration. Ensure they are run before committing by installing pre-commit.

Running Tests

use tox to run all tests across all supported python versions:

pip install tox
tox

Dependencies

Under the hood grand-challenge-forge uses:

Click
- a composable command line interface toolkit
Cookiecutter
- a utility that creates projects from project templates

DIAGNijmegen / rse-grand-challenge-forge

readme