After merging #13, I was surprised about CI, as it did not complain about the output figures. Well, it turns out that the tests were not being properly executed... The command to run the images is tox -e py3X-images not tox -e py3X!
The time plotters are very susceptible to machine processes and other sources of error which can slightly affect the final output. That is why we provide a N_samples, so the problem is computed several times and the final mean values are taken as the "ideal" ones.
As opposite to the time plotters, the number of iterations ones always return the same input, as they only depend on the initial conditions. These are tested using the pytest-mpl plugin.
Therefore, we must find a proper way to test time plotters in order to not lose code coverage.
💻 Feature request: properly test time plotters
After merging #13, I was surprised about CI, as it did not complain about the output figures. Well, it turns out that the tests were not being properly executed... The command to run the images is
tox -e py3X-imagesnottox -e py3X!The time plotters are very susceptible to machine processes and other sources of error which can slightly affect the final output. That is why we provide a
N_samples, so the problem is computed several times and the final mean values are taken as the "ideal" ones.As opposite to the time plotters, the number of iterations ones always return the same input, as they only depend on the initial conditions. These are tested using the pytest-mpl plugin.
Therefore, we must find a proper way to test time plotters in order to not lose code coverage.