Merging crystal maps with not_indexed points results in removal of phase

[hakonanes]

Discussed in https://github.com/pyxem/kikuchipy/discussions/636

^{Originally posted by **Erlendos12** April 10, 2023} I have been trying out kikuchipy 0.8.4 which now keeps the not_indexed points after refinement. However now that I try to merge refined crystal maps which contains not_indexed points, all points in that crystal map will now become not_indexed. E.g. I do hough indexing and refinement on a Super duplex stainless steel with ferrite and austenite. After Hough indexing, I create maps from the data: ```python xmap_austenite = kp.indexing.xmap_from_hough_indexing_data( data=data, phase_list=phase_list, data_index=0, # Phase ID for austenite in phase_list navigation_shape=nav_shape, step_sizes=(1.0, 1.0), scan_unit="um", ) xmap_ferrite = kp.indexing.xmap_from_hough_indexing_data( data=data, phase_list=phase_list, data_index=1, # Phase ID for ferrite in phase_list navigation_shape=nav_shape, step_sizes=(1.0, 1.0), scan_unit="um", ) print(xmap_austenite, "\n") print(xmap_ferrite) ``` ``` Phase Orientations Name Space group Point group Proper point group Color 0 3600 (100.0%) austenite Fm-3m m-3m 432 tab:blue Properties: fit, cm, pq, nmatch Scan unit: um Phase Orientations Name Space group Point group Proper point group Color -1 405 (11.2%) not_indexed None None None w 1 3195 (88.8%) ferrite Im-3m m-3m 432 tab:orange Properties: fit, cm, pq, nmatch Scan unit: um ``` Note here that the austenite map is fully indexed, but not the ferrite one. I do refinement of orientations on each crystal map, and the number of orientations for each phase are the same as before, as expected. However when merging the refined crystal maps which contains not_indexed in the ferrite crystal map, the merged crystal map is completely missing ferrite. ```python refined_xmap_austenite = s.refine_orientation( xmap=xmap_austenite, detector=det, master_pattern=mp_austenite, energy=20, # kV navigation_mask=None, trust_region=[1, 1, 1], # The following are default values method="minimize", method_kwargs=dict(method="Nelder-Mead", tol=1e-4), compute=True, ) refined_xmap_ferrite = s.refine_orientation( xmap=xmap_ferrite, detector=det, master_pattern=mp_ferrite, energy=20, # kV navigation_mask=None, trust_region=[1, 1, 1], # The following are default values method="minimize", method_kwargs=dict(method="Nelder-Mead", tol=1e-4), compute=True, ) print(refined_xmap_austenite) print(refined_xmap_ferrite) refined_xmap = kp.indexing.merge_crystal_maps( [refined_xmap_austenite, refined_xmap_ferrite] ) print("MERGED MAP", refined_xmap) ``` ``` ... Phase Orientations Name Space group Point group Proper point group Color 0 3600 (100.0%) austenite Fm-3m m-3m 432 tab:blue Properties: scores, num_evals Scan unit: um Phase Orientations Name Space group Point group Proper point group Color -1 405 (11.2%) not_indexed None None None w 1 3195 (88.8%) ferrite Im-3m m-3m 432 tab:orange Properties: scores, num_evals Scan unit: um MERGED MAP Phase Orientations Name Space group Point group Proper point group Color 0 1429 (39.7%) austenite Fm-3m m-3m 432 tab:blue 1 2171 (60.3%) not_indexed None None None w Properties: scores, merged_scores Scan unit: um ``` I have also seen that when both refined crystal maps contains not_indexed points, it results in a crystal map where all points are not_indexed. The same also happens without refining the crystal maps before merging (e.g. using `scores_prop = "fit"`) if they contain not_indexed points. Could this have been overlooked in the release of 0.8.4?

---

Thank you for reporting this issue, @Erlendos12. It seems like I introduced a regression in the fix in 0.8.4. Will have a look.

The same also happens without refining the crystal maps before merging (e.g. using scores_prop = "fit")

Note that a lower pattern (mis)fit is better, so you have to pass merge_crystal_maps(..., greater_is_better=False, scores_prop="fit") in this case.

[Erlendos12]

Note that a lower pattern (mis)fit is better, so you have to pass merge_crystal_maps(..., greater_is_better=False, scores_prop="fit") in this case.

Thanks, I forgot to add that.