gdevenyi
commented
10 months ago With regards to the differences, I expect these are numerical, but I'll examine again after you merge into a single welford function that we're not introducing any numerical instability.
As for benchmarking, in the future you can avoid instrumenting the code by running with:
/usr/bin/time -v <command> which gives you summaries of cpu/user/walltime and memory usage of a command.
This PR uses welford algo to fix #78
There are a few branches that I have created for testing memory usage. You can see the results below, and see the branch with its respective code implementation.
For the welford library branch I use the pip Welford library.
The welford algo is an implementation based on this pseudocode.
Ram Usage from 10 MRI scans to get mean
Image Output
Original Code
Welford Algo
Welford Library
As mentioned by @gdevenyi, there should not be any difference between two mean image calculations. This was verified by running the original code and diffing the images.
However, the Welford library and Welford algo had differences when diffing from the original image. I suspect these differences are caused by different initial positions in the x an y axis.
Welford Library vs Original Code
Welford Algo vs Original Code
Original Code vs Original Code
Welford Algo vs Welford Library
Conclusion
If there is a way to verify that the original code and the Welford algo produce the same resultant image, the Welford algo is a good solution to reduce RAM usage.