JeffersonLab / HDGeant4

Geant4 simulation for the GlueX experiment
4 stars 4 forks source link

Calorimeter timing mismatch between g3 and g4 #93

Closed aaust closed 3 years ago

aaust commented 5 years ago

I generated 2 samples of rho events with the trigger emulation turned off, but I still see a large difference in the determined acceptance as a function of the beam energy: image

This is likely due to a very different track timing resolution observed in both calorimeters. Here are example plots for pulls in the FCAL: image image and the difference between reconstructed and thrown pions in the BCAL: image

This is likely related to #89 , even though the FCal shower energy distributions for matched tracks do not look that much different. They do for the BCal. image image

sdobbs commented 5 years ago

It would be interesting to compare the BCAL shower shape variables between HDGeant and HDGeant4 (e.g. DBCALShower::sigLong, DBCALShower::sigTheta...)

aaust commented 5 years ago

Is there a plugin for that? I kept the REST files for both versions.

sdobbs commented 5 years ago

The info is in the REST files.

On Thu, Feb 28, 2019 at 4:12 PM Alex Austregesilo notifications@github.com wrote:

Is there a plugin for that? I kept the REST files for both versions.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/JeffersonLab/HDGeant4/issues/93#issuecomment-468440177, or mute the thread https://github.com/notifications/unsubscribe-auth/ABIJaj1YjsppcleAhulfckeoHyKwRQBpks5vSEY3gaJpZM4bXlSm .

rjones30 commented 5 years ago

I looking into this difference in timing, and I see a difference in the way the times in the BCal hits are being calculated in hdgeant and hdgeant4.

Because these two algorithms are so different, there is no surprise that the resulting histograms do not look the same for the BCal. By contrast, both simulations use the "energy-weighted average" hit time in the FCal. Only in the BCal are they different.

Neither of these two options seems to accurately represent the leading edge time of a pulse. We may want to go to a better algorithm that does not correspond to either of the two that are currently implemented in hdgeant or hdgeant4.

To begin with, I will repeat the above studies using both algorithms in both simulations, a total of 4 combinations. The intention here is to prove that the two simulations give the same results for the hit times in the BCal when they use the same definition of the leading edge.

aaust commented 5 years ago

I just repeated the same study with a recent software stack version 4.5.0. I still see the same discrepancies in the overall acceptance for rhos produced at low energies: acceptance_4 5 0

The obvious differences between track timing pulls in the BCal and FCal are still present as well.

aaust commented 5 years ago

The timing distriution for charged tracks in both calorimeters looks very different:

G3_G4_pion_PID G3_G4_pion_PID_BCal

The timing cut for PID likely causes the difference in acceptance. The timing distributions for data look symmetric, more similar to hdgeant4.

rjones30 commented 5 years ago

Alex,

Can you add a sentence of explanation of what you mean by "very different" in each case? In the FCal, I think you mean the high-side tail in G3 that is less pronounced in G4? In the timing distributions in the BCAL, I think I am not seeing what you do.

-Richard

On Wed, Jul 3, 2019 at 11:18 AM Alex Austregesilo notifications@github.com wrote:

The timing distriution for charged tracks in both calorimeters looks very different:

[image: G3_G4_pion_PID] https://user-images.githubusercontent.com/8807112/60602904-b507b000-9d82-11e9-89e8-bf03860ee279.png [image: G3_G4_pion_PID_BCal] https://user-images.githubusercontent.com/8807112/60603207-4d059980-9d83-11e9-82e8-c86b98aba1b5.png

The timing cut for PID likely causes the difference in acceptance. The timing distributions for data look symmetric, more similar to hdgeant4.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/JeffersonLab/HDGeant4/issues/93?email_source=notifications&email_token=AB3YKWGWTPONXSHVZZYSOCDP5S7NTA5CNFSM4G26KSTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZEZH3Q#issuecomment-508138478, or mute the thread https://github.com/notifications/unsubscribe-auth/AB3YKWHATO5CD5Z2TL5GCMLP5S7NTANCNFSM4G26KSTA .

aaust commented 5 years ago

The plotted deltaT is the difference between the vertex time minus the rf time, propagated to the same vertex.

The timing distributions for both BCal and FCal are broader and asymmetric in G3, with a tail towards positive values.

aaust commented 4 years ago

As promised in the last Geant4 meeting, I made the comparison of the delta t distribution between Geant3, Geant4 and Data. Everything is still about the low energy 2018-08 data set. The plots are normalized by their integral: bcal_comparison While the BCal clearly favors Geant4, the high side of the FCal distribution looks more realistic in Geant3. fcal_comparison

aaust commented 4 years ago

BTW, most of the pions in this sample have a momentum around 2GeV and a polar angle just above 10 degrees. pim_kinematics

zihlmann commented 4 years ago

I kind of would state that geant4 better describes the data event for the FCAL IF the resolution would be correct. There is no "tail" to positive times in the data it just happens that the tail of the geant3MC matches the low resolution of the real data at positive deltaT. if one would smear the geant4 MC FCAL it would very much likely match the data. Or turn the argument around, why is the FCAL timing resolution so bad?

mashephe commented 4 years ago

To add to this...

I would be cautious about introducing a smearing to the FCAL timing based on a study done with charged pions alone. One needs to verify that the timing is appropriate primarily for photons first and then, if possible, pions.

It is likely that smearing alone is not sufficient to get pion timing right (or when implemented it makes the photon time resolution too poor). To get timing right for hadrons one likely has to model, for example interactions with the light guide in order to get the appropriate shapes and tails in timing distributions.

In general the timing characteristics of hadrons are complicated. This is detailed on slide 15 of this talk:

https://halldweb.jlab.org/DocDB/0034/003427/002/fcal_13oct2017.pdf

The plot comes from omega events where one can clearly separate various flavors of photon and hadron-induced clusters. You see for the primary hadron clusters a non-trivial correlation between energy and time. Slide 12 of the same talk shows that this correlation is likely linked to hitting the light guide. Until we model that interaction correctly, I don't know that we are going to get the timing tails right in the simulation for charged hadrons.

As far as I know, Richard has put the light guide into the geometry as an active material. I don't know that energy deposition there propagates up to mcsmear. One would need to get light guide energy and time in mcsmear first. Then one needs to work on an algorithm that takes block energy and time and light guide energy and time and produces the correct response channel energy time for both hadrons and photons.

The sample of events to do this with is exclusive omega events, and one really needs hit-level data to do it correctly -- it should be done on the block level first and then checked at the cluster level. At one time we were skimming these events with every reconstruction launch. I anticipate this is a 6-12 mo. project for a graduate student who is semi-familiar with the GlueX code already. We should really be sure we need an accurate model of tails of hadron times in the FCAL before launching on this project.

Matt

On Nov 13, 2019, at 9:10 AM, zihlmann notifications@github.com wrote:

I kind of would state that geant4 better describes the data event for the FCAL IF the resolution would be correct. There is no "tail" to positive times in the data it just happens that the tail of the geant3MC matches the low resolution of the real data at positive deltaT. if one would smear the geant4 MC FCAL it would very much likely match the data.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

sdobbs commented 4 years ago

@aaust -

The BCAL agreement looks good - do you plan to show this at the meeting this afternoon?

For the FCAL, it would be useful to look at this in a few bins of p-theta. Maybe we can get some other people to look at this as well. I think that this is an argument to start by loosening our standard cuts on the FCAL timing for charged particles if the timing distribution is really that wide. Luckily in the forward direction, the TOF gets most of the charged particles, and as Matt points out, data/MC matching in this case is Not Easy (tm).

@mashephe -

Indeed the light guide is in HDGeant but not propgated through in mcsmear because the required studies haven't been done. Though from Alex's distributions, it seems like we are losing events on both sides of the distributions. Probably we can get away for now with tuning event selections. There are some people who will be interested in a more detailed simulation of these effects though (e.g. the CPP folks).

rjones30 commented 4 years ago

On Nov. 13, 2019 Matt Shepherd wrote:

.. Until we model that interaction correctly, I don't know that we are going to get the timing tails right in the simulation for charged hadrons.

Yes, I agree. There needs to be a detailed study of the timing distribution for hadronic showers from charged pions. Just smearing the timing to match the data is not in view here.

As far as I know, Richard has put the light guide into the geometry as an

active material. I don't know that energy deposition there propagates up to mcsmear. One would need to get light guide energy and time in mcsmear first. Then one needs to work on an algorithm that takes block energy and time and light guide energy and time and produces the correct response channel energy time for both hadrons and photons.

Yes, the energy deposited in the light guide, and its time characteristics, are being stored under a tag distinct from the hits in the lead glass in the hddm output from hdgeant/hdgeant4. AFAIK this information is still being ignored by mcsmear, so it is not being folded into the overall response of the fcal at this point. Doing this requires the study I referred to above.

-Richard Jones

On Wed, Nov 13, 2019 at 9:48 AM Matthew Shepherd notifications@github.com wrote:

To add to this...

I would be cautious about introducing a smearing to the FCAL timing based on a study done with charged pions alone. One needs to verify that the timing is appropriate primarily for photons first and then, if possible, pions.

It is likely that smearing alone is not sufficient to get pion timing right (or when implemented it makes the photon time resolution too poor). To get timing right for hadrons one likely has to model, for example interactions with the light guide in order to get the appropriate shapes and tails in timing distributions.

In general the timing characteristics of hadrons are complicated. This is detailed on slide 15 of this talk:

https://halldweb.jlab.org/DocDB/0034/003427/002/fcal_13oct2017.pdf

The plot comes from omega events where one can clearly separate various flavors of photon and hadron-induced clusters. You see for the primary hadron clusters a non-trivial correlation between energy and time. Slide 12 of the same talk shows that this correlation is likely linked to hitting the light guide. Until we model that interaction correctly, I don't know that we are going to get the timing tails right in the simulation for charged hadrons.

As far as I know, Richard has put the light guide into the geometry as an active material. I don't know that energy deposition there propagates up to mcsmear. One would need to get light guide energy and time in mcsmear first. Then one needs to work on an algorithm that takes block energy and time and light guide energy and time and produces the correct response channel energy time for both hadrons and photons.

The sample of events to do this with is exclusive omega events, and one really needs hit-level data to do it correctly -- it should be done on the block level first and then checked at the cluster level. At one time we were skimming these events with every reconstruction launch. I anticipate this is a 6-12 mo. project for a graduate student who is semi-familiar with the GlueX code already. We should really be sure we need an accurate model of tails of hadron times in the FCAL before launching on this project.

Matt

On Nov 13, 2019, at 9:10 AM, zihlmann notifications@github.com wrote:

I kind of would state that geant4 better describes the data event for the FCAL IF the resolution would be correct. There is no "tail" to positive times in the data it just happens that the tail of the geant3MC matches the low resolution of the real data at positive deltaT. if one would smear the geant4 MC FCAL it would very much likely match the data.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/JeffersonLab/HDGeant4/issues/93?email_source=notifications&email_token=AB3YKWAGLCFIFTNGJ3JNVY3QTQHSDA5CNFSM4G26KSTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOED6MC3I#issuecomment-553435501, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB3YKWF45W2PTLHFG3OJZD3QTQHSDANCNFSM4G26KSTA .

markdalton commented 4 years ago

Here's a plot of the extracted timing offsets from the FCAL for Spring 2017.

FCAL_DeltaTVsRun_201701_recon_ver03

mashephe commented 4 years ago

Mark,

Those jumps appear to correspond to changes in the FCAL HV when we were attempting gain balancing.

Increasing HV reduces the transit time of the electron avalanche in the PMT. You can see slide 13 of this talk from 2009 -- the change is at the order of 1 ns per 100 V.

https://halldweb.jlab.org/DocDB/0011/001198/001/fcal_jan_09.pdf

Naively, I would expect that these modifications would just be random variations about a fixed average, but the shift of pi0 peak described by Adesh in the logbook around that time suggests there were some systematic movements of average HV. I think this is a plausible explanation for the effects that you observe at the 100 ps level (consistent with average voltage change at the 10V per base level).

If it is desirable to remove this, I would recommend a modification of the global FCAL timing for these various run ranges to remove these jumps. Although, I suspect this is is only relevant if we opt to reprocess the data.

Matt

On Nov 13, 2019, at 5:20 PM, dalton notifications@github.com<mailto:notifications@github.com> wrote:

Here's a plot of the extracted timing offsets from the FCAL for Spring 2017.

[FCAL_DeltaTVsRun_201701_recon_ver03]https://user-images.githubusercontent.com/13302551/68809327-b5ba2800-0639-11ea-9322-0738df284e66.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/JeffersonLab/HDGeant4/issues/93?email_source=notifications&email_token=ADG5L2GVQ6AJ3ULK5ROHFZTQTR4URA5CNFSM4G26KSTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOED735VY#issuecomment-553631447, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADG5L2GK2BVO7DQ5DKI7TIDQTR4URANCNFSM4G26KSTA.

markdalton commented 4 years ago

Matt, Your mechanism is certainly interesting although the timing is also influenced by things going on in other detectors and the overall RF timing constant (which is what makes it difficult for more than 1 person to work on timing in our scheme.) My motivation for posting the plot was to demonstrate the consistent difference between charged and neutral particle timing. We must remember that while we are discussing the charged particle performance of the FCAL here, it is primarily used for neutral particle timing.

aaust commented 4 years ago

I made a test, opening the BCal timing cut for pions and protons to from 1ns to 2.5ns. g3_g4_acceptance_openCuts The curves are closer together but still don't agree. Only if I open up the cut to 5ns, the curves are on top of each other: g3_g4_acceptance_openCuts_5ns

With a cut of 5ns, tracks from neighboring beam bunches are included in the sample. Unsing such a wide cut is no feasible for real data. GEANT3 g3_bcal_timing_5ns GEANT4 g4_bcal_timing_5ns

rjones30 commented 4 years ago

Alex,

This represents real progress, as it locates the source of the discrepancy. Thank you for looking into this.

-Richard J.

On Fri, Jan 3, 2020 at 1:45 PM Alex Austregesilo notifications@github.com wrote:

I made a test, opening the BCal timing cut for pions and protons to from 1ns to 2.5ns. [image: g3_g4_acceptance_openCuts] https://user-images.githubusercontent.com/8807112/71741816-d3cc2980-2e2d-11ea-90cd-7aab0a1a2735.png The curves are closer together but still don't agree. Only if I open up the cut to 5ns, the curves are on top of each other: [image: g3_g4_acceptance_openCuts_5ns] https://user-images.githubusercontent.com/8807112/71741892-0fff8a00-2e2e-11ea-956a-2ca45c9b4b3c.png

With a cut of 5ns, tracks from neighboring beam bunches are included in the sample. Unsing such a wide cut is no feasible for real data. GEANT3 [image: g3_bcal_timing_5ns] https://user-images.githubusercontent.com/8807112/71742145-cbc0b980-2e2e-11ea-985b-941b890a9855.png GEANT4 [image: g4_bcal_timing_5ns] https://user-images.githubusercontent.com/8807112/71742150-cd8a7d00-2e2e-11ea-9497-d380c1d9852a.png

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/JeffersonLab/HDGeant4/issues/93?email_source=notifications&email_token=AB3YKWH4PXMAEDCZAJW6S3DQ36BUVA5CNFSM4G26KSTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIBZVLA#issuecomment-570661548, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB3YKWCUU3UHZYJQMBORQY3Q36BUVANCNFSM4G26KSTA .

aaust commented 4 years ago

The second plot in my previous comment was heavily modified by the bug in the analysis library (https://github.com/JeffersonLab/halld_recon/issues/257). Here is the corrected plot for a 5ns timing window: g3_g4_acceptance_openCuts_5ns_bugfix The efficiencies still agree better, but do not drop when widening the cut.

s6pepaul commented 4 years ago

I did a similar study for my channel. I produced flat MC for gp->Kp + L(1520->Kp Km p with a more or less realistic t-slope. So the Kp is very forward going and the proton and Km have fairly low momentum and end up in the central region. I used version set recon-2017_01-ver03_12.xml to run the MC and reconstruction and then switched to a new version of halld_recon with Alex's fix and loosened the PID timing cuts in ReactionFilter. Especially the Kminus (low momentum, central region) looks awful in Geant3. That's why I always claimed I need hdgeant4 to get meaningful results.

G3 G3_Km_tall_DeltaT_BCAL G4 G4_Km_tall_DeltaT_BCAL

For more high level comparisons also see https://halldweb.jlab.org/DocDB/0039/003932/002/pres_looserCut.pdf

aaust commented 4 years ago

pim_TOF_lowE Here I compare the timing distributions for pions ins the TOF from hdgeant and hdgeant4 with a data sample. The resolution is reproduced very well, but geant4 seems to have a tail towards positive values. The default analysis cuts at +-0.5ns.

rjones30 commented 4 years ago

Can you show hdgeant(HADR=1) and hdgeant(HADR=4) together with hdgeant4 and data? Remember that there are two variants of hdgeant. If we continue to show plots with hdgeant vs hdgeant4 we lose sight of the fact that hdgeant has 2, not 1 prediction.

-Richard J.

On Mon, May 11, 2020 at 2:02 PM Alex Austregesilo notifications@github.com wrote:

[image: pim_TOF_lowE] https://user-images.githubusercontent.com/8807112/81595065-dfa59a80-938f-11ea-8ccc-e274a8677bb5.png Here I compare the timing distributions for pions ins the TOF from hdgeant and hdgeant4 with a data sample. The resolution is reproduced very well, but geant4 seems to have a tail towards positive values. The default analysis cuts at +-0.5ns.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/JeffersonLab/HDGeant4/issues/93#issuecomment-626861761, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB3YKWEMKH2IB6ZA7WV6S53RRA4U3ANCNFSM4G26KSTA .

markito3 commented 3 years ago

These topics are now being pursued under Issue #111