search

StochasticNumerics / mimclib

A software library for UQ methods
GNU General Public License v2.0
6 stars 8 forks source link

Plotting routine producing a ton of error output as well as erroneous numbers #79

Closed Virtakuono closed 7 years ago

Virtakuono commented 7 years ago

I shall paste all the output I get below:

jhappola@samubo:~/mimclib/tests/gbm$ ./plotexample If all goes well, this script should plot the relevant plots of the parallel mimc run. Output will be produced to 'GBM*.pdf' Plotting GBM_std_real Plotting GBM_arr Plotting GBM_obj Traceback (most recent call last): File "", line 1, in File "/home/jhappola/mimclib/mimclib/plot.py", line 1356, in run_program raise Exception("No runs!!!")

Exception: No runs!!!

Traceback (most recent call last): File "/home/jhappola/mimclib/mimclib/plot.py", line 1069, in genPDFBooklet num_kwargs={'color': 'r'}) File "/home/jhappola/mimclib/mimclib/plot.py", line 249, in plotErrorsVsTOL modifier = (1./fnNorm(np.array([exact]))[0]) if relative_error else 1. File "mimc_run.py", line 130, in run.setFunctions(fnNorm=lambda x: np.max(np.abs(x), axis=1)) File "/usr/lib/python2.7/dist-packages/numpy/core/fromnumeric.py", line 2268, in amax out=out, keepdims=keepdims) File "/usr/lib/python2.7/dist-packages/numpy/core/_methods.py", line 26, in _amax return umr_maximum(a, axis, None, out, keepdims)

ValueError: 'axis' entry is out of bounds

Traceback (most recent call last): File "/home/jhappola/mimclib/mimclib/plot.py", line 1079, in genPDFBooklet Ref_kwargs={'ls': '--', 'c':'k', 'label': '{rate:.2g}'})
File "/home/jhappola/mimclib/mimclib/plot.py", line 440, in plotWorkVsMaxError
exact=exact)
File "/home/jhappola/mimclib/mimclib/plot.py", line 313, in computeIterationStats
modifier = (1./fnNorm(np.array([exact]))[0]) if relative else 1.
File "mimc_run.py", line 130, in
run.setFunctions(fnNorm=lambda x: np.max(np.abs(x), axis=1))
File "/usr/lib/python2.7/dist-packages/numpy/core/fromnumeric.py", line 2268, in amax
out=out, keepdims=keepdims)
File "/usr/lib/python2.7/dist-packages/numpy/core/_methods.py", line 26, in _amax
return umr_maximum(a, axis, None, out, keepdims)

ValueError: 'axis' entry is out of bounds

Traceback (most recent call last):
File "/home/jhappola/mimclib/mimclib/plot.py", line 1090, in genPDFBooklet
Ref_kwargs={'ls': '--', 'c':'k', 'label': '{rate:.2g}'})
File "/home/jhappola/mimclib/mimclib/plot.py", line 440, in plotWorkVsMaxError
exact=exact)
File "/home/jhappola/mimclib/mimclib/plot.py", line 313, in computeIterationStats
modifier = (1./fnNorm(np.array([exact]))[0]) if relative else 1.
File "mimc_run.py", line 130, in
run.setFunctions(fnNorm=lambda x: np.max(np.abs(x), axis=1))
File "/usr/lib/python2.7/dist-packages/numpy/core/fromnumeric.py", line 2268, in amax
out=out, keepdims=keepdims)
File "/usr/lib/python2.7/dist-packages/numpy/core/_methods.py", line 26, in _amax
return umr_maximum(a, axis, None, out, keepdims)

ValueError: 'axis' entry is out of bounds

plot.py:942: (UserWarning) QQ plots require the object to implement float

Virtakuono commented 7 years ago

GBM_std_real.pdf The resulting plots are also of questionable quality, at least in the time versus level, the point estimates of the time consumed are outside their respective confidence bands.

haji-ali commented 7 years ago

Can you redo the runs into an SQLite database and send me the file? The many errors are because you only have data for GBM_std_real you have to generate data for GBM_arr and GBM_obj to see the other plots.

Virtakuono commented 7 years ago

I'm still finishing the first run, do you have a ballpark estimate on how long the parallel run example should take on your former laptop? Can I interrupt it without messing up the database?

Virtakuono commented 7 years ago

Ooops, closed by fat finger.

haji-ali commented 7 years ago

You can always interrupt and the database should always remain consistent. Try to plot and if the the data is already showing the wrong behaviour just interrupt the simulation and send me the data file.

Virtakuono commented 7 years ago

Okay, here comes: https://dl.dropboxusercontent.com/u/12915479/mimc

haji-ali commented 7 years ago

Fixed and pushed the changes to the dev branch.

I decided to push all these bug fixes to the dev branch (incrementing the dev version number) and then merge into the master branch when I am more sure that things are stable.

Regarding the bug. I was plotting the mean instead of the median which gave strange results when the mean was above the 0.95 percentile (that's the extent of the error bars).

I also added to the time plot a line showing the "wall clock". This is the time that also includes the book-keeping of the MLMC algorithm; previously only the sampling time was plotted. I am still trying to figure out why the wall clock is very different from the sampling time for small tolerances.

haji-ali commented 7 years ago

I didn't realize that you had changes in master. I merged now those with dev. Let us try to stick to dev -> master kind of merges from now on.

Virtakuono commented 7 years ago

Ookay, I was just pushing things to the first thing I saw. I'll try to keep pushing to dev from now on. Sorry for the hassle.

Virtakuono commented 7 years ago

Awesome. Seems to work now using https://github.com/StochasticNumerics/mimclib/commit/e79d951d6e634e70c33c6564d86214e2fc5419d4, attaching the new plots for the sake of completeness. GBM_std_real.pdf