Binary form, code and so on

[stefanmengel]

Some questions on how the authors should provide the tools/solvers:

Solvers should be available in binary form for research purposes

Maybe you should rather say that they should be made available without prescribing the form in the headline and then discuss details in the text? If there is a (compilable) version of the source code, do you still want to have a binary?

Papers presenting experimental results about a system (solver) should allow the community to access this solver in at least binary form, or even better in source form. This allows to check performance claims independently.

You might want to add that if the source is given, then there should also be instructions on how to compile, dependencies and so on. The source alone won't help anyone if no one can compile it.

(No idea if I am doing this github thing right...)

[kuldeepmeel]

I agree; How about

Solvers should be available for research purposes

Papers presenting experimental results about a system (solver) should allow the community to access this solver so as to allow independent verification of performance claims. The authors are strongly suggested to do an open source release of their solvers; In circumstances where open source release is infeasible, the executables should be made publicly available.

[DvonHolten]

just an idea for discussion: would it make sense to encourage the solver-implementors not just to explain their latest and greatest innovations, but also to explain, why their newest thing is noticibly bad on some benchmarks ? (if so)

[kuldeepmeel]

I agree but It is hard to codify as an abiding principle because lot of times, there are no good answers.

[arminbiere]

Yes, I suggest to close this issue.