Indexing of patterns in H5OINA files

[hakonanes]

Moving here a short discussion started in https://github.com/EMsoft-org/EMsoft/issues/164 regarding reading EBSD patterns stored in H5OINA files in EMsoft. @Philip-Go kindly shared a small H5OINA file (136 MB), which I could confirm that the soon to be released kikuchipy reader could read successfully (including the pattern center values):

Quoting @Philip-Go:

That might be off topic regarding EMsoft, but the resulting IPf map looks well indexed. Does PyEBSDIndex achieve such a high indexation rate by hough indexing alone or was the map subjected to some cleaning and filtering afterwards? I'm wondering, since with online indexing during the measurement AZtec yields only ~60% indexation rate. I should definately take a closer look at that, thanks showing that!

The IPF map is produced from the results returned from PyEBSDIndex using default values. Do you have anything to add @drowenhorst-nrl? Feel free to close this issue when it suits you.

---

Below I've included the script I used. It requires the development version of kikuchipy, so to install everything (remaining packages are dependencies of kikuchipy):

pip install git+https://github.com/pyxem/kikuchipy.git@develop
pip install pyebsdindex[gpu]

Remove the [gpu] if an error message shows (see the installation guide). Indexing will use CPU so will be quite a lot slower, but still fast (I'd say). An unimportant warning is raised, which is fixed and will be released as a PyEBSDIndex v0.1.1 patch soon.

Then this should produce the plot above:

import os

from diffpy.structure import Lattice, Structure
from diffsims.crystallography import ReciprocalLatticeVector
import numpy as np
from orix import io, plot
from orix.crystal_map import CrystalMap, Phase, PhaseList
from orix.quaternion import Rotation

import kikuchipy as kp
from pyebsdindex import ebsd_index

# Load data and set some shapes
dir_data = "/home/hakon/phd/data/oxford"
fname = "Al_SLBM 10kV_8x8_small"
s = kp.load(os.path.join(dir_data, fname + ".h5oina"))
nav_shape = s.axes_manager.navigation_shape[::-1]
sig_shape = s.axes_manager.signal_shape[::-1]

# Create indexer and perform Hough indexing
indexer = ebsd_index.EBSDIndexer(
    vendor="KIKUCHIPY",
    sampleTilt=s.detector.sample_tilt,
    camElev=s.detector.tilt,
    patDim=sig_shape
)
data, *_ = indexer.index_pats(
    s.data.reshape((-1,) + sig_shape), PC=s.detector.pc.reshape((-1, 3))
)

# Store results in a crystal map and save in an .ang file
rot = Rotation(data[-1]["quat"])
phase = Phase(
    "al",
    space_group=225,
    structure=Structure(lattice=Lattice(0.404, 0.404, 0.404, 90, 90, 90))
)
y, x = np.indices(nav_shape)
xmap = CrystalMap(
    rotations=rot,
    x=x.flatten() * s.axes_manager["x"].scale,
    y=y.flatten() * s.axes_manager["y"].scale,
    phase_list=PhaseList(phase),
    prop=dict(
        pq=data[-1]["pq"],  # Pattern quality
        cm=data[-1]["cm"],  # Confidence metric
        fit=data[-1]["fit"],  # Pattern fit
        nmatch=data[-1]["nmatch"],  # Number of detected bands matched
    ),
)
io.save(os.path.join(dir_data, fname + "_hi.ang"), xmap)

# Inspect results using geometrical simulations
ref = ReciprocalLatticeVector(
    phase=phase, hkl=[[1, 1, 1], [2, 0, 0], [2, 2, 0], [3, 1, 1]]
)
ref = ref.symmetrise().unique()
simulator = kp.simulations.KikuchiPatternSimulator(ref)
sim = simulator.on_detector(s.detector, xmap.rotations.reshape(*xmap.shape))

# Create IPF-Z map
ckey = plot.IPFColorKeyTSL(phase.point_group)
rgbz = ckey.orientation2color(xmap.orientations).reshape(nav_shape + (3,))
map_rgbz = kp.draw.get_rgb_navigator(rgbz)

# Navigate patterns in IPF-Z map with geometrical simulations
s.plot(navigator=map_rgbz)
s.add_marker(sim.as_markers())

Here are some further tutorials of Hough indexing with PyEBSDIndex:

• https://pyebsdindex.readthedocs.io/en/latest/tutorials/ebsd_index_demo.html
• https://kikuchipy.org/en/latest/tutorials/hough_indexing.html

[hakonanes]

This check also serves as a confirmation that the following route of PC value conversion works: Oxford Instruments -> kikuchipy/Bruker -> PyEBSDIndex.

[drowenhorst-nrl]

May thanks for putting that together. I guess I have assumed that OXFORD PC == EDAX PC, and thus that direct input of the Oxford PC was not a necessary test. I tried pulling that smaller datafile, but the link is dead already (at least for me). But I really like the integration that you did with kikuchipy. I think the differences I am seeing in the EMsoft IPF and Kikuchipy are likely because you did not specify the camera tilt, and thus using a default value. No big deal, but if anyone asks, this is the likely answer.

Also nice to see that it works well with wide band patterns, that can be harder for Hough-based methods. But what is going on with AZtech? 60% indexing on what looks like nice patterns? Wonder if some of the parameters are not set properly. I will reply in the EMsoft thread expressing my gratitude for the data/validation.

[hakonanes]

I think the differences I am seeing in the EMsoft IPF and Kikuchipy are likely because you did not specify the camera tilt, and thus using a default value.

You're right. In the imminent new minor release of kikuchipy (v0.8), the detector tilt (and binning) is read with the patterns from H5OINA files. The detector parameters are stored in the EBSD.detector attribute of the returned signal.

Taking the detector tilt into account, the IPF-Z map looks like this after HI + orientation refinement using dynamical simulation from EMsoft (dynamic PCs were extrapolated from a mean assuming a detector pixel size of 25 um [as per Philip] and the geometric relations from the appendix of the refinement paper by Singh et al. (2017)):

Colors are quite similar to the maps in https://github.com/EMsoft-org/EMsoft/issues/164#issuecomment-1289151978.