search

olexandr-konovalov / gnu

Crowdsourcing project for the database of numbers of isomorphism types of finite groups
MIT License
11 stars 4 forks source link

R-words: getting the terminology right #73

Open olexandr-konovalov opened 8 years ago

olexandr-konovalov commented 8 years ago

Currently we use "validated" label to mark gnu(n) which we were able to confirm and orange for those awaiting validation. As suggested to me by Bettina Eick, we should have more colors to distinguish the cases when the number has been determined by at least two independent means, and when has been determined by just one method. She adds that "In all cases, it needs to be recorded carefully which method(s) of determination have been used. It would also be highly useful if a number is checked against the literature."

I agree - we need to distinguish better between different degrees of validation, and get our usage of "R-words" right (you may find discussions and references at https://github.com/ReScience/ReScience-article/issues/2 and https://github.com/ReScience/ReScience-article/issues/5 useful).

I suggest to use the following system:

For the numbers checked against the literature, I suggest a separate label (in some green areas of the spectre) called agrees with theory.

Does this sound good?

olexandr-konovalov commented 8 years ago

A related question is then when to consider the issue or pull request to be closed? My answer would be: when the value of gnu(n) is added to the database, that is when it is "replicated". Thus the workflow of this project is not only to move issues from open to closed, but also from "replicated" to "reproduced".

markuspf commented 8 years ago

The database should contain all this data itself, as of it being replicated in GAP, the value for gnu(n) goes in the database, and is tagged as "replicated" (maybe including version numbers, things like the GAP repositories commit hash?). The value can then gain the "reproduced" or "agrees with theory" tag plus references if and when this happens to be the case.

markuspf commented 8 years ago

With references I also mean that if there is a proof for a theorem that justifies the value, we then link to the literature.

Better yet, if we produce proofs ourselves, we could try and write them up and store them inline with the database as a "live document" (And, as a pipe-dream at some point one could attempt to put together a more classical paper from the collected writings)

olexandr-konovalov commented 8 years ago

Now I have added the label "agrees with theory" to #22. I suggest to keep "reproduced" for data which are computed using software, so one could have a combination of both "agrees with theory" and "reproduced" labels as well.

olexandr-konovalov commented 8 years ago

My plan is to add new section "Reliability of the data" to the README.md file and close this issue afterwards.