search

Benchmarking-Initiative / Benchmark-Models-PEtab

A collection of mathematical models with experimental data in the PEtab format as benchmark problems in order to evaluate new and existing methodologies for data-based modelling
BSD 3-Clause "New" or "Revised" License
26 stars 14 forks source link

Checklist for new submissions #206

Closed dilpath closed 4 months ago

dweindl commented 4 months ago

Hm, why a new file instead of updating/referencing https://github.com/Benchmarking-Initiative/Benchmark-Models-PEtab/blob/master/.github/PULL_REQUEST_TEMPLATE/new_petab_problem.md?

dilpath commented 4 months ago

Hm, why a new file instead of updating/referencing https://github.com/Benchmarking-Initiative/Benchmark-Models-PEtab/blob/master/.github/PULL_REQUEST_TEMPLATE/new_petab_problem.md?

Good point, I'll try to consolidate them. I don't see this template when I open a new PR, hopefully moving it to pull_request_template.md fixes that

dweindl commented 4 months ago

Good point, I'll try to consolidate them.

:+1:

I don't see this template when I open a new PR, hopefully moving it to pull_request_template.md fixes that

That's weird. I think it was working at some point.

dilpath commented 4 months ago
  • [ ] Differences between the implementation and the original publication are described

Should that be in the issue? It might be preferable to have it in the SBML annotations?

  • [ ] There is a GitHub issue for this problem

Do we really need an issue? It makes sense if one starts to convert some published problem to PEtab, to avoid other people working on the same thing. But if somebody wants to submit their just published files, I don't think we need an issue for that.

  • [ ] Source of nominal parameters (e.g.: taken from the original publication, or from your own fitting)

SBML annotation instead of gh issue?

At the moment, we don't have a nice place to put metadata -- this might change with the PEtab Result format, but we will probably still need some YAML that has other metadata.

For example, I don't think annotating the SBML with the reference DOI is a good idea, since this collection may contain PySB and other valid "PEtab v2" models soon. For the same reason, I also think things like model ID should rather exist in the PEtab YAML -- i.e. the model should not be relied upon by PEtab for metadata. So, some YAML with reference DOI etc. makes most sense to me -- or we integrate this information directly into the PEtab YAML. I guess we could design a simple extension for the PEtab YAML in PEtab v2, for users to specify metadata like reference DOI and arbitrary notes, what do you think?

dweindl commented 4 months ago

Agreed that it makes sense to have the info on the dataset, provenience information on nominal parameters, ... in the PEtab files, because it doesn't directly affect the SBML model.

I know that at some point we had a discussion on whether PEtab yaml file should allow adding additional values (either arbitrary additional entries, or at least something like annotations:). I can't find a github issue on that, and I don't remember the conclusion. But if we allow that, we could easily add that information there, according to a custom schema defined here. It wouldn't really have to be a PEtab extension.

That's a different discussion, though. So ignore those comments for the purpose of this PR.