Right now the notification that you get via submission doesn’t tell you that this will create a PR that will be tested. Need to add that with a link to the PR to track the status.
Nice to have: Check to see if there is a PR for the given model ID and don’t double submit.
Add-on: Check through PRs and remove any existing duplicates. Can't have a cluttered development environment.
Make sure email method actually works (core, each repo has a submissions dir, and an endpoints.py that handles communication between repos) https://github.com/brain-score/core/blob/ec8d7eb6ff3598e82b58917fa1708172f2c569ad/brainscore_core/submission/endpoints.py#L74C9-L74C24 Travis could be failing, too. Check that to be safe https://github.com/brain-score/vision/blob/master/.travis.yml https://github.com/brain-score/core/blob/main/brainscore_core/travis/submission_failure.yml
Right now the notification that you get via submission doesn’t tell you that this will create a PR that will be tested. Need to add that with a link to the PR to track the status.
Nice to have: Check to see if there is a PR for the given model ID and don’t double submit.
Add-on: Check through PRs and remove any existing duplicates. Can't have a cluttered development environment.