Ideas here is to use the Zenodo versioning to version our benchmarks rather than use our arbitrary fallible system. We submit each notebook to Zenodo and then update the same series of data with the new notebook when changes are made. Each version of the notebook will have a DOI and also there will be a DOI associated with all the versions.
The submission schema in pfhub-schema should point at the version DOI as well as having human readable benchmark number + zenodo version number. Benchmark versions are 1a.zenodo_doi, but mostly referred to as 1a for example. Get rid of the 1a.0 1a.1 etc. It's just confusing for people
Authors should be based on those that have edited the notebook and had names on the paper associated with the benchmark.
How to cite this benchmark should be at the top of each benchmark notebook
Ideas here is to use the Zenodo versioning to version our benchmarks rather than use our arbitrary fallible system. We submit each notebook to Zenodo and then update the same series of data with the new notebook when changes are made. Each version of the notebook will have a DOI and also there will be a DOI associated with all the versions.
The submission schema in pfhub-schema should point at the version DOI as well as having human readable benchmark number + zenodo version number. Benchmark versions are 1a.zenodo_doi, but mostly referred to as 1a for example. Get rid of the 1a.0 1a.1 etc. It's just confusing for people
Authors should be based on those that have edited the notebook and had names on the paper associated with the benchmark.
How to cite this benchmark should be at the top of each benchmark notebook