Correct setting of the U matrix for sample misalignment correction

[azkhadiev]

Hi @dkriegner ,

first of all thank you very much for your amazing software. At the moment I have faced a problem with correcting the sample position with a U matrix. I have measured three reflections in cubic system: 003, 103 and 113 and suddenly has found that sample was tilting during the temperature change of the sample. I suggested that I can use 003 and 103 reflections (they were almost at the same plane during the measurement, so I was able to change only angle indicent to the sample and detector angle while switching between them) to correct the sample tilt as you described in your example (xrayutilities/examples/xrayutilities_orientation_matrix.py). But I have found that after getting U matrix from 003 and 103 reflections and aplying it to 113 reflection, 113 reflection position is not at 45 degree to the X axis. There was a shift of around 0.88 degree. Do you know what can cause the problem? Here is our analysis sequence: 1. define the diffractometer and measurement conditions for 003 reflection:

qconv = xu.experiment.QConversion(['y-','x-','z-'], ['y-'], [1,0,0]) hxrd = xu.HXRD([1, 0, 0], [0, 0, 1], geometry="real", qconv=qconv, en=energy)

hxrd.Ang2Q.init_area('y+',
                 'z-',
                 cch1=cen_pix[0],
                 cch2= cen_pix[1],
                 Nch1=sample_set['frames'].shape[1],
                 Nch2=sample_set['frames'].shape[2],
                 pwidth1=5.5000e-05,
                 pwidth2=5.5000e-05,
                 distance=det_distance,
                 roi=[imgroi[1].start,imgroi[1].stop,imgroi[2].start,imgroi[2].stop])

2. prepare the gridder: bins = (400, 400, 1500) gridder_al = xu.FuzzyGridder3D(*bins)

3. Apply the data qx, qy, qz = hxrd.Ang2Q.area(sample_set['omega'], chi, phi, sample_set['delta'], en=energy) gridder_al(qx, qy, qz, sample_set['frames'][imgroi]) qdata_al = gridder_al.data
   3. Find the 003 reflection position: qsym = peak_position_sym(qdata_al, gridder_al) tz = xu.math.AxisToZ_keepXY(qsym)

3. Use this tz for finding the U from 103 reflection. The data processing of the 103 reflection is the same as for 003 relfection instead of the last step:

gridder_al(qx, qy, qz, sample_set['frames'][imgroi]) qdata_al = gridder_al.data qasym = peak_position_sym(qdata_al, gridder_al) qasym_tiltcorr = tz(qasym) phicorr = np.arctan2(qasym_tiltcorr[1], qasym_tiltcorr[0]) ta = xu.math.ZRotation(-phicorr, deg=False) U = np.linalg.inv(np.dot(ta.matrix, tz.matrix))

4. Apply the U matrix for 113 reflection calculation. Detector, diffractometer and experiment definition is the same as in previous steps, except the UB:

qx, qy, qz = hxrd.Ang2Q.area(sample_set['omega'], chi, phi, sample_set['delta'], en=energy, UB=U)

Finally I see that the position of the 113 peak is not at 45 to the X axis but at 45.88 degree. What do you think can cause such a error? I could imagine that I have to use another hxrd = xu.HXRD([1, 0, 0], [0, 0, 1], geometry="real", qconv=qconv, en=energy) definition in case of 113 reflection. It is not clear for me how the UB selection interferes with the sample position definition expressed in the hxrd = xu.HXRD() method. In the calculation I used real omega, chi, phi and detector angles without any offsets. The RSM of 003 and 103 reflections contrary to 113 look reliable.

Thank you for your time and consideration.

Regards, Azat.

[dkriegner]

Dear @azkhadiev

Let me try to attempt to answer your questions:

But I have found that after getting U matrix from 003 and 103 reflections and aplying it to 113 reflection, 113 reflection position is not at 45 degree to the X axis. There was a shift of around 0.88 degree. Do you know what can cause the problem?

as far as I can see the procedure you apply seems correct. I also deeply hope there are no flaws in my example (but you can convince yourself by looking at every piece of the code). Potential problems could be in the detector calibration. If the detector pixel rows are for example not perfectly aligned with the y/z directions. This effect would be minimized if the same detector area is used to measure all individual peaks. Also if you use a goniometer with multiple rotation cradles mounted on top of each other their zero positions need to be accurately aligned so that the theoretical description of the goniometer (in your case ['y-','x-','z-']) really applies. I have found this to ofter not be very accurate. For many measurements this does not play a very crucial role, but in your case here could induce this found unexpected difference.

In this respect I am also wondering what you mean by the following:

In the calculation I used real omega, chi, phi and detector angles without any offsets. The RSM of 003 and 103 reflections contrary to 113 look reliable.

What does it mean that 003 and 103 look reliable? From the procedure above you force the 003 peak to have be aligned with the z-axis, and the 103 peak can have only one inplane component. But since you have a cubic system also their z-component would need to perfectly agree? Is this the case? Any difference here could have a similar origin as mentioned above.

I could imagine that I have to use another hxrd = xu.HXRD([1, 0, 0], [0, 0, 1], geometry="real", qconv=qconv, en=energy) definition in case of 113 reflection.

That is for sure not the case. The same hxrd object should be applicable for all measurements done with the same goniometer. There is a longstanding problem here with the logic of the code. The first three arguments here in fact only affect the Q2Ang conversion and have no influence to the Ang2Q conversion on which your application is based.

I hope this comments help. Its hard to remotely analyze such problems

[azkhadiev]

Dear @dkriegner,

thank you very much for your explicit answer.

What does it mean that 003 and 103 look reliable? From the procedure above you force the 003 peak to have be aligned with the z-axis, and the 103 peak can have only one inplane component. But since you have a cubic system also their z-component would need to perfectly agree? Is this the case? Any difference here could have a similar origin as mentioned above.

z component of 003 and 103 reflections was not 100 % equal. The error between them was about 0.02 %. I thought that it can be assosiated with the error of peak position determination (I used the scipy.ndimage.center_of_mass function).

Thank you for your advice to do proper detector calibration. I didn't use all parameters, achieved by xu.analysis.sample_align.area_detector_calib() method. It seeems that I didn't fully understand what data can be used for detector calibration. During the experiment I performed the mesh scan of the direct beam using two 'perpendicular' motors (gamma and delta) and got around 140 points of the direct beam on the detector. Can I use these data if I put correct values of ang1 and ang2 (gamma and delta) in xu.analysis.sample_align.area_detector_calib() method? Or only the two separate scans with two motors are possible for getting the right calibration: ang1 scan (ang = 0 during this scan) and ang2 scan (ang1 = 0 during this scan) as you described in your Acta paper?

I will try to get better calibration of the detector and see the results.

Thank you again for your help!

[dkriegner]

A mesh scan should be perfectly fine for the detector calibration. The results of the calibration have to always be checked carefully, but I would hope would help with your problem.

[azkhadiev]

Dear dkieger,

yes, it is a bit hard for me to check whether the calibration is good. It seems to me that I do something wrong... Does it matter for you how your software 'see' the detector image? I think that there are, lets say, two options: to see the image from the sample side or to see the image from the backside of the detector (these images are mirrored in respect to each other). Is this parameter also defined somewhere in your software? The second question is about the goniometer definition. In your Acta article, you define the goniometer by (‘z-’, ‘x-’, ‘y+’), but the phi rotation is counter-clockwise according to your Fig. 2, so I would expect (‘z-’, ‘x-’, ‘y-’). Is it a typo simply?

Thank you for your tiem and consideration. Sorry for bothering you again.

[dkriegner]

Does it matter for you how your software 'see' the detector image? I think that there are, lets say, two options: to see the image from the sample side or to see the image from the backside of the detector (these images are mirrored in respect to each other). Is this parameter also defined somewhere in your software?

Its matters of course. But the init_area function's first two arguments in my opinion uniquely define the detector's pixel row directions because the laboratory coordinate system is used as a basis. (so how you look at does not matter). That being said: I also often need to fiddle around with this to get it right. One has to be very careful here because some image plotting functions have the (0,0) pixel not bottom right, but somewhere up, which I think is often the origin of the confusion.

The second question is about the goniometer definition. In your Acta article, you define the goniometer by (‘z-’, ‘x-’, ‘y+’), but the phi rotation is counter-clockwise according to your Fig. 2, so I would expect (‘z-’, ‘x-’, ‘y-’). Is it a typo simply?

Indeed this seems to be a misprint. Clearly phi is "y-" (equal to the direction of the angle delta which is also listed as "y-" in the paper), I would nevertheless call it clockwise since because it rotates clockwise around y (note that y points backwards).

[azkhadiev]

Yes, I also thought that init_area function uniquely defines the detector position by itself independent of the image rotations/inversions... But I have tried different image orientations of the image before calling init_area function and sometimes results are more reliable, like this:

and sometimes I see a huge dispersion of the data:

Is it because there lot of local minima in the fit? You said that you sometimes have to fiddle around to get it right. Do you have some kind of parameter that shows you that the alignment is right?

I also have found that it is also possible to use the area_detector_calib_hkl method. Is it more robust?

Thank you again for your answers!

[dkriegner]

I would say if your original issue is solved then you know you got it right :) I know this is little help, but its hard to say more from my perspective.

From the two solutions you sent I agree that the first one seems better. The epsilon parameter is much lower, although I have seen already much lower values in general. But this depends mostly on how precisely the primary beam position can be identified by the script. This can be judged when switching on the debug flag and looking at the images which should be saved then (one image per detector frame).

and yes its true that this fit seems to have many local minima. I never found a better way as to start from a range of input parameters which seems reasonable. On thing I want to point out in your case is that detrot is far from zero and clearly should not be ignored in the final data evaluation. The outer angle offset is a more complicated issue and usually has the highest fit error, but could in your case be at least contributing to the original issue.

Be sure you understand the last line of the text printed by the fit output!

the method with _hkl usually improves the situation with the outer angle offset. I consider it to be more robust, but not many people have used it so far. so again be careful with the results.

[azkhadiev]

Ok. Thank you for your help. In my case I check whether the angle between 003 and 103 reflections in the cubic system is close to the table values after I do detector alignment with different inputs. This parameter should be independent of the X-ray wavelength, so only the detector alignment parameters have an influence on this value (assuming that the diffractometer is working correctly). I don't know whether such parameters can also be used for fitting of you, just wanted to express some ideas.

[dkriegner]

indeed this idea came already from many people. I think if one has some NIST certified cubic material one could do this. If I ever completely rewrite this part of the code I shall remember it.

[azkhadiev]

I just tried to do a calibration with different initial parameters and use an 'eps' as a key parameter showing the quality of calibration.

It seems that in the function:

param, eps = xu.analysis.sample_align.area_detector_calib()

eps (returned parameter) is not the epsmin found by the algorithm. The eps is not the same epsilon found in the text protocol after the calibration. But the param are attributed to the fit with min eps, right?

[dkriegner]

You found a real issue here. The fit did return the last epsilon it found but not the minimal one. The parameters were, however, the correct ones. The mentioned commit (c2b0ecf) fixes the return value.