ValueError when removing static background

[jorgenasorhaug]

The following error is printed when I try to remove the static background on a signal with original file extension .up2. How can I solve it?

-----------------------------------------------------------------------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
File ~\Anaconda3\envs\kp_ebsd\lib\site-packages\kikuchipy\signals\ebsd.py:357, in EBSD.remove_static_background(self, operation, static_bg, scale_bg, show_progressbar)
    356     if not isinstance(static_bg, (np.ndarray, da.Array)):
--> 357         raise ValueError
    358 except (AttributeError, ValueError):

ValueError: 

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
Cell In [16], line 1
----> 1 s.remove_static_background()

File ~\Anaconda3\envs\kp_ebsd\lib\site-packages\kikuchipy\signals\ebsd.py:359, in EBSD.remove_static_background(self, operation, static_bg, scale_bg, show_progressbar)
    357             raise ValueError
    358     except (AttributeError, ValueError):
--> 359         raise ValueError("`EBSD.static_background` is not a valid array")
    360 if isinstance(static_bg, da.Array):
    361     static_bg = static_bg.compute()

ValueError: `EBSD.static_background` is not a valid array

[hakonanes]

Hi Jørgen,

this specific error is raised because the EBSD.static_background attribute is not a valid array. It should be None if it is not.

Might your patterns already have the static background removed? I believe patterns are always stored with the static background removed by EDAX by default. At least the patterns in UP2 files I've encountered before. If not, was there some static background stored with the UP2 file? If yes, you can pass that to EBSD.remove_static_background(). If not, you can create an average pattern and use that:

>>> static_bg = s.mean(axis=(0, 1))
>>> static_bg.change_dtype(s.data.dtype)
>>> s.remove_static_background(static_bg=static_bg.data)

[jorgenasorhaug]

The background was removed in the software used on the instrument, but when looking into the up2 file in kp it didn't seem to be the case.

How do I check if there are any stored background in the signal?

[hakonanes]

Could you show a raw pattern here?

The reader of UP1/2 files in kikuchipy is based on the reader in PyEBSDIndex written by @drowenhorst-nrl (source). I don't know whether there is a static background pattern written to the binary file, and if so, in which bytes it is located.

[drowenhorst-nrl]

For what it is worth, and to the best of my knowledge- the up1/2 files from EDAX will save with whatever image processing steps are applied in the live collection. Or in other words, what you see on the camera preview image in OIM Collection/Teams/Apex is what you get in the up1/2 files. Thus, if collected with background subtraction on, I don't know of a way to remove the background subtraction after the fact.

This is because EDAX does not (at least up to version 3 of the files) keep a separate background image. I have seen that there is a version 4 of the up1/2 files, but have yet to see the changes documented, and when I have just applied the current V3 reader, it looks to me like I get the patterns I expect.

If you want to have raw patterns without a background removal, you can remove all image processing steps before collection, and that is what it will save. The live indexing will likely be very poor in this case.

[jorgenasorhaug]

I might be wrong then if it is the case that the image processing step done in Apex EBSD is already applied to the signal.

Any idea if the removed background is the static and dynamic one? The software only calls it background ..

[hakonanes]

EDAX will save with whatever image processing steps are applied in the live collection

Makes sense, thank you for clarifying, @drowenhorst-nrl.

Any idea if the removed background is the static and dynamic one?

It seems to me like the static background is removed. From my experience with EDAX TSL Data Collection v7.2, EDAX applies a radial mask to patterns when patterns are visualized in the software, so that everything apart from the pattern within the aperture is black. It might also be that they then adjust the contrast, which might make it appear different in the software.

Not a perfect pattern, but it seems to me like the bands are clear enough for Hough indexing. The uneven intensity should be removed with a call to EBSD.remove_dynamic_background().

[drowenhorst-nrl]

I think @hakonanes has it all correct. The default is to use a static background subtract (dynamic is quite slow) and the radial mask is applied. There are options for adjusting contrast digitially, but on the Hikari cameras, it was expected that the gain adjustment on the camera would be used to get the contrast correct. On the velocity cameras, there is no grain, thus it is sometimes useful to apply it digitally.

Not sure what PyEBSDIndex would do with low-contrast patterns ... I don't see why that would affect anything. But I know EDAX will change its values with digial contrast adjustment on/off. This might have something to do with the "Min peak magnitude" adjustment on the hough, which there is no equivalent for in PyEBSDIndex.

[hakonanes]

@jorgenasorhaug, feel free to close the issue if your questions have been answered.

Not sure what PyEBSDIndex would do with low-contrast patterns

For completeness: kikuchipy supports several routes for intensity adjustments, e.g. simple rescaling to fill the data type range ([0, 255] for unsigned 8-bit) or with/without clipping of brightest/lowest intensities, etc. The relevant tutorial has more details.

[jorgenasorhaug]

Thanks for all the help :)

[hakonanes]

No problem. Please continue to open issues when you encounter them!