Welcome to the ICLR 2022 Computational Geometry & Topology challenge 2022 --- by the ICLR 2022 Workshop on Geometrical and Topological Representation Learning.
Lead organizers: Adele Myers, Saiteja Utpala, and Nina Miolane (UC Santa Barbara).
The purpose of this challenge is to foster reproducible research in geometric (deep) learning, by crowdsourcing the open-source implementation of learning algorithms on manifolds. Participants are asked to contribute code for a published/unpublished algorithm, following Scikit-Learn/Geomstats' or pytorch's APIs and computational primitives, benchmark it, and demonstrate its use in real-world scenarios.
Each submission takes the form of a Jupyter Notebook leveraging the coding infrastructure and building blocks from the package Geomstats. The participants submit their Jupyter Notebook via Pull Requests (PR) to this GitHub repository, see Guidelines below.
In addition to the challenge's prizes, participants will have the opportunity to co-author a white paper summarizing the findings of the competition.
This is the second edition of this challenge! Feel free to look at last year's guidelines, submissions, winners and paper for additional information.
Note: We invite participants to review this README regularly, as details are added to the guidelines when questions are submitted to the organizers.
The final Pull Request submission date and hour will have to take place before:
The participants can freely commit to their Pull Request and modify their submission until this time.
The first 3 winners will be announced at the ICLR 2022 virtual workshop Geometrical and Topological Representation Learning and advertised through the web. The winners will also be contacted directly via email.
The prizes are:
Anyone can participate and participation is free. It is enough to:
An acceptable PR automatically subscribes a participant to the challenge.
We encourage the participants to start submitting their Pull Request early on. This allows to debug the tests and helps to address potential issues with the code.
Teams are accepted and there is no restriction on the number of team members.
The principal developpers of Geomstats (i.e. the co-authors of Geomstats published papers) are not allowed to participate.
A submission should respect the following Jupyter Notebook’s structure:
torch.nn.Module
.geomstats.backend
are not considered computational primitives, as they are only wrappers around autograd, numpy, torch and tensorflow functions.Before starting your implementation, make sure that the algorithm that you want to contribute is not already in the learning module of Geomstats.
The notebook provided in the submission-example-*
folders is also an example of submission that can help the participants to design their proposal and to understand how to use/inherit from Scikit-Learning, Geomstats, Pytorch. Note that this example is "naive" on purpose and is only meant to give illustrative templates rather than to provide a meaningful data analysis. More examples on how to use the packages can be found on the GitHub repository of Geomstats.
The code should be compatible with Python 3.8 and make an effort to respect the Python style guide PEP8. The portion of the code using geomstats
only needs to run with numpy
or pytorch
backends. However, it will be appreciated by the reviewers/voters if the code can run in all backends: numpy
, autograd
, tensorflow
and pytorch
, using geomstats gs.
, when applicable.
The Jupyter notebooks are automatically tested when a Pull Request is submitted. The tests have to pass. Their running time should not exceed 3 hours, although exceptions can be made by contacting the challenge organizers.
If a dataset is used, the dataset has to be public and referenced. There is no constraint on the data type to be used.
A participant can raise GitHub issues and/or request help or guidance at any time through Geomstats slack. The help/guidance will be provided modulo availability of the maintainers.
Fork this repository to your GitHub.
Create a new folder with your team leader's GitHub username in the root folder of the forked repository, in the main branch.
Place your submission inside the folder created at step 2, with:
Datasets larger than 10MB shall be directly imported from external URLs or from data sharing platforms such as OpenML.
If your project requires external pip installable libraries that are not amongst Geomstats’ requirements.txt, you can include them at the beginning of your Jupyter notebook, e.g. with:
import sys
!{sys.executable} -m pip install numpy scipy torch
The Condorcet method will be used to rank the submissions and decide on the winners. The evaluation criteria will be:
Note that these criteria do not reward new learning algorithms, nor learning algorithms that outperform the state-of-the-art --- but rather clean code and exhaustive tests that will foster reproducible research in our field.
Selected Geomstats maintainers and collaborators, as well as each team whose submission respects the guidelines, will vote once on Google Form to express their preference for the 3 best submissions according to each criterion. Note that each team gets only one vote, even if there are several participants in the team.
The 3 preferences must all 3 be different: e.g. one cannot select the same Jupyter notebook for both first and second place. Such irregular votes will be discarded. A link to a Google Form will be provided to record the votes. It will be required to insert an email address to identify the voter. The voters will remain secret, only the final ranking will be published.
Feel free to contact us through GitHub issues on this repository, on Geomstats repository or through Geomstats slack. Alternatively, you can contact Nina Miolane at nmiolane@gmail.com.