Open mrmhodak opened 7 months ago
Thank you @mrmhodak for formulating this. I would like to add the below point too.
Any submitter can request for access to a submission system to reproduce the submission. If the submitter grants the access and in case the reviewer is satisfied with his reproducibility work, this process can be used to waive (need not be binding) the system from the official audit.
Arjun, that is an interesting proposition. I will bring it up for discussion in the WG leadership first.
Any submitter can ask for access to a submission system to reproduce the submission.
Very interesting idea. One clarifying question here: "can ask" implies that the submitter asked can say "no.". I think clarifying right of refusal here is helpful in evaluating the language. Is it true that submitters can say "no" when asked?
Any submitter can ask for access to a submission system to reproduce the submission.
Very interesting idea. One clarifying question here: "can ask" implies that the submitter asked can say "no.". I think clarifying right of refusal here is helpful in evaluating the language. Is it true that submitters can say "no" when asked?
Thank you @bitfort Yes, the submitter can say no. I just edited the comment to make this clear.
I have simplified the text per @bitfort.
Also, added @arjunsuresh suggestion on granting access and avoiding audit
@bitfort : I would like to merge this before Review Meetings start. Excluding the last clause about audit, does the rest look good to you?
Pushed an update to remove the waiving audit clause. Too close to submission deadline to get an agreement.
To make it easy to find, the final version of the waiving audit clause was:
During the Peer Review Process, a reviewer can ask for access to a submission system to reproduce the submission. If the request is granted and the reviewer is satisfied with the results, this can be used to waive the reviewed system from the audit, subject to the approval of the review group
.
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅