pyxem / kikuchipy

Toolbox for analysis of electron backscatter diffraction (EBSD) patterns
https://kikuchipy.org
GNU General Public License v3.0
79 stars 30 forks source link

Use HyperSpy's map() to reduce time and memory use in some EBSD processing methods #527

Closed hakonanes closed 2 years ago

hakonanes commented 2 years ago

Description of the change

This PR introduces many internal changes and some external ones.

The improved map() method in HyperSpy v1.7 is used in some EBSD processing methods, reducing memory use and the processing time! HyperSpy's minimal version is set to 1.7. The map() method does not print a progressbar even though we ask for one (see https://github.com/hyperspy/hyperspy/pull/2947), so for now we add this manually. This leads to two bars for some reason, but two are better than none.

The above changes made the following kikuchipy.pattern.chunk functions unnecessary, and have hence been removed from the API (in v0.6): remove_static_background(), remove_dynamic_background(), get_image_quality().

The EBSD.remove_static_background() parameter relative, to maintain relative intensities between patterns during background removal, has been removed. It can still be passed, but a deprecation warning is raised if it is. Passing it after v0.7 is released will raise an error. This was done because the (bad) implementation required that all patterns had to be queried for the minimum and maximum intensity value before background removal, effectively touching every pattern chunk twice. Dask could for some reason handle this well previously, but not later versions (don't know exactly which versions), and lead to large memory use for large datasets. I deem keeping relative intensities between patterns an unimportant feature in kikuchipy, and hence have dropped the functionality entirely in one go.

I've added formatting of notebooks using black's Jupyter notebook extension (thanks to @harripj for mentioning it in orix!). This is also added to the CI workflow code style check.

FFTs are computed using functions from scipy.fft instead of numpy.fft.

Progress of the PR

Minimal example of the bug fix or new feature

These operations are faster since they use hyperspy.signals.BaseSignal.map():

>>> import kikuchipy as kp
>>> s = kp.data.nickel_ebsd_large()
>>> s.remove_static_background()
Removing the static background:
[########################################] | 100% Completed |  0.1s
[########################################] | 100% Completed |  0.1s
>>> s.remove_dynamic_background()
Removing the dynamic background:
[########################################] | 100% Completed |  0.1s
[########################################] | 100% Completed |  0.6s
>>> iq = s.get_image_quality()
Calculating the image quality:
[########################################] | 100% Completed |  0.1s
[########################################] | 100% Completed |  0.3s

For reviewers