stan-dev / posteriordb

Database with posteriors of interest for Bayesian inference
176 stars 36 forks source link

Add tests and gold standard systems information #116

Closed MansMeg closed 4 years ago

MansMeg commented 4 years ago

Diagnostics:

In addition to the checks, also add slots in the gold standard that all these things have been checked for explicit in the gold standard.

Additional gold standard information: R/Python version, the makevars file, the systemInfo() (and equivalent for Python)

eerolinna commented 4 years ago

Which things exactly does the equivalent for python need to include?

MansMeg commented 4 years ago

So if we use python to compute a Gold standard we need to know the Python version, versions of additional libraries, and makevars? I guess there is a way to identify the ”state” that was run?

eerolinna commented 4 years ago

I don't think python has anything that is equivalent to R makevars. However I'm not fully sure if I understand what makevars are. Is it that makevars specifies the "environment variables" for building R packages?

Would we need versions of only the libraries that were actually used in the code that computes the gold standard or would it suffice to just include versions of all installed libraries?

What state are you referring to in the last sentence? Do you mean the things you mentioned in the previous sentence or something else?

MansMeg commented 4 years ago

I think the makevar file defines how the C++ compiler is used. Im not sure but I think it is used when compiling the Stan C++ code.

I guess all installed libraries would do. It is more for reproducibility that for actual use.

Yes, the state is the "current state of your Python and Stan implementations".

MansMeg commented 4 years ago

Yes, makevar is the compiler flags.

Also, some functions to access Divergences etc can be found in bayesplot.

Also, add indicators for checked diagnostics.

MansMeg commented 4 years ago

Now fixed.