Multi-histogram data sets producing inaccurate asymmetry spectra

[benfrandsen]

Describe the bug When combining histograms to produce an asymmetry spectrum, the resulting spectrum shows significant distortions, particularly at long time.

To Reproduce Steps to reproduce the behavior:

1. Add the attached data file from PSI to a BEAMS session.
2. Convert the data file using histograms 1 and 5 as the "front" detector and 3 and 7 as the "back detector".
3. Load and plot the data out to 8 us.
4. Two problems appear: (a) The spectrum blows up beyond about 7 us. (b) The asymmetry doesn't oscillate around a constant; the tops and the bottoms of the wiggles do not converge uniformly as would be expected for typical exponential decay.

Expected behavior The correct asymmetry as generated by PSI's software is shown in the screenshots below.

Screenshots Correct asymmetry:

Incorrect asymmetry generated by BEAMS showing how it blows up beyond 7 us:

Incorrect asymmetry generated by BEAMS showing how the spectrum doesn't oscillate uniformly around a constant.

Note that if we use just histograms 1 and 3 to generate the asymmetry in BEAMS (i.e. we don't combine any histograms), then the spectrum does oscillate around a constant, as expected. This suggests that the problem arises from not correctly combining the histograms. However, spectrum still blows up beyond 7 us, so that is likely a separate problem. Here's the BEAMS asymmetry using just histograms 1 and 3:

Desktop (please complete the following information):

• Windows
• Master branch
• Python 3.10.9

Additional context The correct way to generate the asymmetry when combining multiple histograms is shown below:

[benfrandsen]

Here's the data file you can use for testing: lem17_his_2162.zip

[aPeter1]

I think there was a misunderstanding around combining histograms on my part, I assumed if you were using 1 and 3 for the forw, then you could combine those into a single histogram (and then same process for the back) and then create the asymmetry the normal way using the two combined histograms. But it appears you keep them separate, but use them together to calculate the asymmetry correct?

[benfrandsen]

That was my understanding, too, and I still feel like it should work. But our asymmetry with combined histograms is clearly distorted somehow, so it seems we need to try it differently. Let me see if I can get some clarification from the beamline scientists.

[alec-petersen]

Sounds good, I'll try doing it both ways (outside of beams) to see how the results vary

[benfrandsen]

Some information from instrument scientist Zaher Salman:

Combining two histograms is quite simple. You need to align the t0's of the histograms then add the counts channel by channel before and after t0. The histograms will all have the same time resolution but will most probably have different t0's. This could potentially cause the distortions you mentioned.

In the case of bulk instruments at PSI (except for HAL) you can simply take t0 from the channel with maximum counts (prompt peak). In the case of HAL and LEM you are better off relying on the saved t0's in the data file.

[aPeter1]

Niiiiice definitely wasn’t lining them up like that before so that’s one problem. Tell him thanks!!

[aPeter1]

I pushed a fix for this one as far as being centered; let me know if it looks better to you. Still have other issues.

[aPeter1]

Ah, if you checkout the histogram tab in beams, you can see background end is way further ahead then it should be (we had to calculate start and end manually because its not in the files). Wonder if that will be in the new musr root files. I'll take a look.

EDIT noooooo they still aren't including it for some reason (or I'm misreading the docs)

EDIT EDIT on the other hand, adjusting the background does make it look better so that's a plus. I'll change how we set those two to be much more conservative.

[aPeter1]

I'm going to adjust our previous method and say 10% from the first non zero value to t0 and 40% before t0 to the first non zero value. That gives us this which seems closer to ideal (and the asymmetry looks waaaaaaay better). I think all issues are solved, but let me know otherwise.

[aPeter1]

Well. There still is one issue actually, the start of the asymmetry doesn't look the same as the graphic you shared (it looks like a piece is cut off...) Is the file you sent to me the same one you used to generate the images (from beams and PSI) above?

Also, can t0 and first good bin be the same? Using the python library that's what I'm getting out and that seems odd. Shouldn't t0 be a bit before? (in this case, around 220?)

Also, the app does not look good on a 4k screen. Which is weird because I though I got rid of all hard limits on sizing.

[benfrandsen]

Thanks, I see a lot of improvements. I think the background was the main issue giving us trouble with the distortions. However, as you point out, we now have the problem that the asymmetry generated by beams seems to be out of phase from how it should be, e.g. starting out near zero instead of near its maximum. This is strange, because it wasn't happening before we fixed the issues with the background and t0. It seems to me that this could happen if one of the histograms has the wrong t0 value. Could you try to reverse engineer what's causing this problem? I can also look into it by playing around with the individual histograms.

I also agree that it's weird for t0 and goodbin1 to be the same. That doesn't seem right to me. It seems like the values in the header maybe aren't reliable.

[benfrandsen]

By the way, the out-of-phase issue persists even when NOT combining multiple histograms, so it's not caused by something you did when combining histograms. This is very strange!

Also, this probably won't be very fun, but I think we need to add in the capability to adjust the background, t0, and good bins for each individual histogram. At the moment, after the histograms are combined upon converting the file, there doesn't seem to be a way to go back and adjust anything with the individual histograms before combining.

[aPeter1]

Well I've added somewhat equivalent functionality that was easier to manage in the short-term. If you click "Don't Combine" this will create a dat file with all the histograms. If you load that and go to the histogram tab you can select some histograms, right-click, and click combine (and specify a title). This will add that histogram to the run and refresh the window with the updated histogram in the list. Then you can update the t0's and good bins and what not and combine again if you need to.

This doesn't add it to the .dat file currently, you would need to save the run again for that to happen. But I could change that if we wanted to.