psychoinformatics-de / paper-remodnav

Code, data and manuscript for https://doi.org/10.1101/619254
Creative Commons Attribution 4.0 International
4 stars 2 forks source link

Recompute analysis results in dockerized environment #24

Closed mih closed 11 months ago

mih commented 11 months ago

The creation of a containerized environment for the analysis became necessary, because 3+ years after the "final" results have been computed originally, it is getting difficult to recreate a matching computational environment.

Even with pinned versions of essential software dependencies, issues of incompatibilities with modern Python versions slowly arise.

The container setup used for this recomputation is the result of a detailed exploration on the effect of software versions and deployment methods. A reports is provided at https://github.com/psychoinformatics-de/paper-remodnav/issues/20

Importantly, the employed setup is NOT capable of yielded exactly identical results. While all statistical scores reported in the paper remain indeed identical, there is a visually small change to one histogram panel in Fig 4. The change is illustrated at https://github.com/psychoinformatics-de/paper-remodnav/issues/20#issuecomment-1757462683

Given the overall state of reproducibility, and the anticipated longevity of the containerized computation, we decided that this small difference with respect to the journal publication is tolerable.

This changeset support a DataLad-based re-execution (for verification):

datalad rerun <commitsha>

After this changeset, a complete manuscript can be compiled, also via DataLad via a:

datalad containers-run -n docker-make main.pdf

Closes #20

TODO: