performance optimizations for wavelets and testing framework in python

[Arcadia197]

Wavelets

Fresh latest batch of performance optimizations for wavelets!

• removed the usage on inflatedMallat and changed all occurrences to work with Spaghetti form, getting rid of copy operations
• mask and norm are only computed the first (or also second) adapt_tree iteration as afterwards they
  • do not change refinement stati for mask check as all mask-affected blocks were already checked
  • do not change norm value to a significant amount as only non-significant details are deleted
• buffer zone could be changed to something closer to $N{WC}$ instead of $N{rec}$, however this change is for now not implemented as we await a feedback from Kai
• wavelet decomposition and reconstruction was cleaned and many copy operations removed. You can now also read both functions and actually understand more easily what is going on

A comment on the last commit as we did not discuss it and I did it on Friday night: While cleaning up I noticed that your definition for the spaghetti form is:

HH HG
GH HH

This means that with changing x-coordinate we change the filter in y-dimension (interchanging between scaling and wavelet). While this is convenient with the naming (SC, WCx, WCy, WCxy), I think it is confusing for the indexing. Additionally it means both in wavelet decomposition and reconstruction we swap the filtered values after (or before) we apply the filters. I would love to change this definition to match the indexing and avoid confusion. I have changed all necessary comments in the code already. Please give me a feedback on this!

Timing

I added timing information for every adapt tree iteration. It may add quite some timing entries but they help currently with understanding what is going on in adapt_tree. Would love to see one for a full test and in near future we could disable them again

Testing framework

New tests need latest python-tools, make sure to update it (and merge the PR there)

Main tests

I have recreated the testing framework in Python. This gives it more flexibility with execution with additional parameters. While at it I tweaked the tests:

• blob2D CDF4X was tweaked to have more than only 4 blocks
• blob3D was changed to a less time-intensive version
• added ACM40 for comparison

Total timing on IDRIS 326s and locally 286s.

Compression test

I have recreated the compression test in python and added it to the main library so that it can be easily deployed on systems as well. It is included in TESTING/compression.

[tommy-engels]

Should all unit tests pass with this version? Or do they have to be updated ?

[Arcadia197]

Should all unit tests pass with this version? Or do they have to be updated ?

All should pass but apparently blob_adaptive_CDF_42 fails and I don't know why currently.

[Arcadia197]

I forgot to mention: The tests now directly access wabbit_tools.py so this needs to be accessible for python in the PYTHONPATH via something like. I added this to the wiki.

export PYTHONPATH=${PYTHONPATH}:${WABBIT_PT_ROOT}