setup continuous integration - Githubissues

brain-score / brainio_contrib

contribute stimuli or data to BrainIO

http://brain-score.org

3 stars 11 forks source link

setup continuous integration #4

Closed mschrimpf closed 4 years ago

mschrimpf commented 5 years ago

My favorite CI providers are:

Travis (remote; worked with it in the past; $69/month for private repos but potentially free for .edu)
Jenkins (self-hosted)
CodeShip (I know the founder, so good support line) see also https://www.g2crowd.com/categories/continuous-integration

There are then three checks until a pull request is merged:

automatic (see below)
Chris: to accept the run
Sachi: after the data has been collected and the assembly is available in Brain-Score

Automatic checks:

no naming conflicts between registered StimulusSets
all registered functions actually create a StimulusSet in the desired format (i.e. pandas DataFrame with at least id column, as well as pointing to an image file that can be loaded through e.g. PIL) notes:
could ignore StimulusSets that are registered and already available on S3

Further points to discuss:

are experimental requests separate from the StimulusSet? are they part of it? should users register both a stimulus set as well as an experimental request?

mschrimpf commented 4 years ago

Done a while ago now