Fix the definition of a TMuonEvent

[benkrikler]

Similar to the discussion we had about TAnalysedPulses, we need to fix the contents of TMuonEvents which goes hand in hand with what we understand this to mean.

[AndrewEdmonds11]

So far, I've been thinking that we would have a TMuonEvent which would have all TDetectorPulses that are within a certain time of a muSc pulse (I guess, a TDP).

A question I have is, would we want different times for different detectors? e.g. Do we want to consider Ge TDPs as part of a TME if it's within 500ns but a muScA as part of a TME if it's within 10ns?

Or do we do all these timings afterwards?

[thnam]

I would like to pick all pulses between two consecutive muSc pulses because the time differences w.r.t. muSc contain some physics.

[benkrikler]

There were two sub-questions that I'd had on this though you've mentioned them already;

1. Do we store more than one pulse per channel in a single TME?
   • One TME has 1 muon and every pulse that followed it
   • One TME has at most 1 pulse per channel and therefore possibly 0 muons in the event
2. In the first of the above options, how do we define the window to collect pulses in?
   • This is the same as Andy's question above I think.
   • I think start window time is set by the TDiff value for that channel which we set in a previous run (or from the LLDQ plots).
   • I think the stop window time is set by the next muon event. That means there's another form of pile-up if physics from one event overlaps with the next muon's arrival. Is this a problem and what can be done?

[litchfld]

[Caveat: i still think my understanding of the measurement is a bit shakey, so this may be all wrong. But let's dive in anyway]

I think the time windows involved have to come from the physics. If I see a time line

MuSc  .........X...............................X
SiR1  ......................X....X.X............
SiR2  .................................X........

I want to know about all those SiR1 pulses, so i can decide whether or not they make it look suspicious. So I think max 1 pulse per channel is not workable, unless the majority of analysis is happening at the TDP stage.

Also I definitely want to know when the next muon is to evaluate my pile up inefficiency. But i don't necessarily want to stop at the next muon if that was close after my event. This may be obvious pile up on the first muon:

MuSc  .........X.............X..................
SiR1  ........................X.................
SiR2  .................................X........

But what about, say, this?:

MuSc  .........X.............X..................
SiR1  ..............X...........................
SiR2  ..........................................

So I probably want to have enough information to do my event selection as normal, and only then look at the time to the next muon.

[AndrewEdmonds11]

I also remember when we talked about this in the past that we might collect the same TDP into two different muon events. Something like this:

muSc  .......x...........x..........
SiR1  .............x................
TME1      |.........|
TME2            |.........| 

[jrquirk]

The TME was born from a muSc centered event. That is, every muon was at the center of a 20 us end-to-end window, so that

we might collect the same TDP into two different muon events

just like the image above. In fact, if two muons were close together, they would each be in each other's events, and each would be the t=0 center of one event.

This was the original idea put forth by Peter Winter, whether or not we stick with it is a separate question. But doing [-10us, muon t=0, 10us] in all detectors means we can analyse each event individually since 10us is a long time (even if the muon stops in plastic).

See Peter Winter's slide 18 here for the original idea.

[benkrikler]

My major concern with reusing TDPs in multiple TMEs is how we prevent against over-counting. A single TDP should only contribute once to a final analysis, how do we ensure that this happens?

My feeling is that the part of the analysis which builds TMEs should be deciding in which TME to place a given TDP. A simple implementation could just take all pulses between consecutive muon arrivals and make a TME out of that. A more complex one could look for TDPs after the next muon's arrival and based on timing and energy information assign that into the previous TME.

For example:

MuSc      .....X...............................X...........................X
SiR1      ..................X....X.X..............X...................X.....
SiR2      .............................X..............X...........X.........
TME_simple[1]  |............1....1.1...2.......|                            
TME_simple[2]                                  |..1...2...........2...1....|
TME_complex[0] .............1|
TME_complex[1] |.................1.1...2..........1...2|
TME_complex[2]                                 |..................2...1....|

My point is that 2 TMEs could overlap in time, but wouldn't be able to reuse a TDP.

It boils down I guess to what part of the analysis we say 'this TDP came from this muon event'? Is it in the process of making a TME or using the TMEs to form the input to this question?

Of course, for some things, like looking for the correlation that gives the TDiff peak, we don't mind over-counting, in which case reused TDPs is not a problem. But for rate estimates and normalisations this would be important.

[jrquirk]

I would like to get a more experienced person in on this conversation. You're double-counting concern makes sense, but then a lot of our measurements are correlations and the TDiff argument applies. Also, having an event-dependent time windows seems like it could introduce biases.

We could attach weights to events based on how many TMEs they're in as well.

[litchfld]

It boils down I guess to what part of the analysis we say 'this TDP came from this muon event'? Is it in the process of making a TME or using the TMEs to form the input to this question?

I think it's after... otherwise you are basically trying to do the the reconstruction on the TDP containers directly, and the TME is just some kind of DST.

As long as the the TME has records of other possible muons, it should be easy to spot when double counting is a possibility. We aren't (shouldn't be) stupid enough to interpret an event where a second muon comes through before anything else happens other as two capture signals, but it would be useful to know how often that happened. We are probably going to throw out both alternatives in such events, but being able to plot distributions helps you understand whether it's accidental co-incidences or something systematic, which you need to do to make an appropriate efficiency correction.

[benkrikler]

@jrquirk:

You're double-counting concern makes sense, but then a lot of our measurements are correlations and the TDiff argument applies.

But if our final measurement is a rate or particle spectrum then we really cannot allow over-counting. When we look for the TDiff value and other correlations that's more of a calibration issue.

Also, having an event-dependent time windows seems like it could introduce biases.

I don't think it would introduce any more bias than fitting some function to a different number of samples in a pulse each time, which is pulse dependent. As long as we're using some fixed definition for how to determine the end of a TME and not going through deciding by hand each time, then I don't think you'd be adding bias.

We could attach weights to events based on how many TMEs they're in as well.

That would work. It would mean adding a field to a TDP to store how many times it's included in a TME. It would also be useful, in terms of later deciding which TME it belongs to the best, if we could also find out which TMEs contained the pulse (as well as how many). This sort of thing breaks the chain of TAP => TDP => TME though, as TDPs now know about TMEs. We could get into messy circular dependencies if we're not careful...

@litchfld:

otherwise you are basically trying to do the the reconstruction on the TDP containers directly, and the TME is just some kind of DST

I don't see the problem with doing reconstruction on the TDPs directly. They're like hits in a straw tracker, why add another intermediate layer? Also, DST?

As long as the the TME has records of other possible muons, it should be easy to spot when double counting is a possibility.

That's not all the information you would need to avoid double counting. You would also need to know if a TDP on any channel has been allocated to another TME as well.

Here's how I've always envisaged a TME to work, just to put it clearly once and for all (though not to say this is the way it should be):

• A TME represents the reconstructed information of 1 muon event in the chamber (so only 1 muSc hit per TME).
• It contains all TDPs that can be associated with this muon event and each TDP is only placed once in a TME (multiple different TDPs per channel per muon event, but no reusing TDPs in other TMEs).
• It forms the final reconstructed level before analysis. After this it's just producing coincidence cuts to get the measured spectra and rates.

I don't think a TME should be a simple rearrangement of the TDPs. If it is, why bother at all, when we could just work with TDPs directly?

[AndrewEdmonds11]

A few points:

• Based on the presentation John posted (and from what I recall), TMEs were just supposed to be a rearrangement of TMEs so that we were centered around some physics.
• Over-counting definitely sounds like a problem which, maybe, wasn't considered before.
• Ben's TME above is certainly how I envisaged the TME to be used (but I didn't really think about having the same TDP in more than one TME)

Maybe we should just send Peter W an e-mail asking him about it...

[benkrikler]

I've emailed Peter, hopefully he can post here, but if not I'll upload the feedback.

[litchfld]

[...] otherwise you are basically trying to do the the reconstruction on the TDP containers directly, and the TME is just some kind of DST

I don't see the problem with doing reconstruction on the TDPs directly. They're like hits in a straw tracker, why add another intermediate layer? Also, DST?

"Data Summary Tape." (FTR I was not around when such things were actually stored on tape). Typically the DST contains enough information to do the analysis, but very little of the reconstruction details.

To take your straw tracker example, it's computationally infeasible to do analysis on tracker hits directly, so there is usually several intermediate stages after that: track finding & fitting; 'event' building; application of veto, fiducial and noise cuts...

A DST might include things like the fitted track charge & momentum, a reconstructed start vertex, and particle ID assignment. It would deliberately not include some things (like maybe the charge of individual hits) because they make the summary too large. Analysis of our hypothetical experiment might be how the particle fractions depend on momentum, as a function of the vertex position (if that sounds daft - it's the basis of some sterile neutrino search proposals, so yes it is daft. ;) ) That can be done with the DST, rather than the full reconstruction output, to ease computation.

My conception of a TME is not like a DST, it's more like the track- or event-building stage. You may well, at that point, have two overlapping physics events. The idea is that you have enough information within the event candidate to decide if it looks like one real physics interaction or something else: maybe it's pile-up you can't make sense of, maybe it's all noise, maybe it's mostly one event with a little bit of noise & or pile up that you are comfortable removing/ignoring.

[litchfld]

I think the over counting problem is being overstated here. Yes it's a problem if the same physics event ends up in more than one physics classification or (less commonly) in the same classification twice. However it's totally okay that a single detector event can veto more than one other (imagine a detector that is partially dead for a long period after an event), so in that sense a single TDP can contribute to the analysis of multiple events.

To put it another way, when we decide on the interpretation we should take into account all the information that is in some kind of causal window of a true event, then decide if all that information together looks characteristic of a single event. The non-observation of additional signals within that window is also important information.

This does open up the possibility of double counting, but the answer then is to either write the logic so that each solution is exclusive, or to calculate the correction factor, which is often pretty easy. The nearest-muon approach is essentially the simplest logic that satisfies the exclusivity approach, but it additionally hides information that may have been relevant to other muons.

[jrquirk]

I want to give an example of why I'm concerned about

so only 1 muSc hit per TME

and why it would be more of a problem than

I don't think it would introduce any more bias than fitting some function to a different number of samples in a pulse each time

If we say each measurement window starts with one muon, and then ends at the next one, our timing windows have a length distributed according to the beam stats, which is exp(-t/T), where t is the length of the timing window and T is something like 1/10kHz. Now we're looking for a proton or electron or something to indicate a capture or decay to fill a lifetime histogram, which should follow an exponential as well. But now, because we have a varying time window decided by the beam, we're taking more measurements in the earlier window of times. This means we're just more likely to see the disappearance of the muon if it happens earlier because most of our measurement windows are going to include the window of time 0-50ns, but only some of them will include the window 650ns-700ns, so even if the decay happens out there we are less likely to measure this.

What this ends up doing is convolving the beam function with the lifetime function. Below is an example

The following script was used to generate this

{
  double beam_tau = 1./10000.;
  double lifetime = 840.e-9;

  beam_tau *= 1.e9.;
  lifetime *= 1.e9.;

  TH1I* hReg = new TH1I("hReg", "Lifetime;Time (ns)", 1000, 0., 10000.);
  TH1I *hCon = new TH1I("hCon", "Lifetime;Time (ns)", 1000, 0., 10000.);

  TRandom3 ran;

  double next, decay;
  for (int i = 0; i < 1000000; ++i) {
    decay = ran.Exp(lifetime);
    next = ran.Exp(beam_tau);
    if (decay < next) {
      hCon->Fill(decay);
      if (next > 10000.) {
    hReg->Fill(decay);
      }
    }
  }
  hReg->SetLineColor(kBlack);
  hCon->SetLineColor(kRed);

  hReg->SetStats(0);
  hCon->SetStats(0);

  TLegend *leg = new TLegend(0.7,0.7,0.9,0.9);
  leg->AddEntry(hReg, "Vetoed if another muon in 10us");
  leg->AddEntry(hCon, "Window determined by next muon");

  hReg->Draw();
  hCon->Draw("SAME");
  leg->Draw("SAME");
}

So one histogram (hCon) counts a disappearance in the lifetime histo if it happens before the next muon. The other only counts it if the next muon does not occur in the 10us window. Clearly there's an effect by eye. By math, the lifetime difference is actually 832 ns vs 838 ns (I put in 840 for the the lifetime for some reason) if fit to the entire histogram, so actually not that bad. Plus the rates we had were lower than 10kHz, which make things better.

Also, at first glance it doesn't seem like this roving window would affect the total rates or energy spectrum (we don't care what daughter came from what muon in those cases I think).

And this doesn't mean we couldn't use the time-window-defined-by-next-muon, we would just need to account for this. I still vote for the original muon centered tree, though.

[thnam]

@jrquirk Thanks a lot for the nice demonstration! I also think that a long enough time window should be okay, there is little going on far from t0:

Even if we use the next muon as the end of time window, I think that we still have to introduce a pile up protection (like in the figure above), or we will see something like this:

[benkrikler]

Peter replied before the meeting but I didn't have time to post it. I see it was discussed anyway from the minutes.

Ben,

I glanced over the long thread and there are good questions in there. I can comment on this during the analysis conference. In short, in former experiemnts we always stored all pulses in a +-N microsecond window around each possible muon. Of course that means that some pulses after one muon might also get assigned to the next muon so one has to deal with the double assignment. One could get smart and only store the pulses once in the tree and pointers to those pulses if they are used again with another muo. But in our case we might not worry too much about that and rather just store them twice or 3 times instead of coming up with some smart linking via pointers to save space. Another issue is the pileup protection which requires a second stream of pileup vetos (typically the OR of all muon counters and vetos) in addition to the muon pulse (typically the AND of all muon entrance counters). Since we do pileup protection in a +- M microsecond window around each muon, we also will need to bookend correctly at the beginning and end of each MIDAS event. That means, the first muon we can really use is M microseconds into the fill andd the last comes M microseconds before it ends.

Peter

[benkrikler]

So just to make sure I'm on the same page as everyone, a TME will :

• Contain multiple hits on ALL channels
• Be defined by a hard window of +-N us (which we would tune) around each muon and collect all hits within that window.
• not overlap with any other TME.

When later using a TME:

• Multiple muons within a window == pile-up and should be rejected.
• Define a muon hit as a hit in all muon counters (muSc && muPC X && muPC Y [ and not muScA ? ] )
• Consider a hit in any of the muon counters as indicative of pile-up (muSc || muPC X || muPC Y ).
• Ignore TMEs that come within N us of the start of a midas event (possibly pile-up).

Does that sum up what was said?

Two questions I have then:

1. What defines the time of a TME?
   • If there's only one muon its obvious, the time of that muon.
   • If two muons fall within the window, do we take the average arrival time, or the first?
2. If three or more muons come each one separated by <N us from the previous one, what do we do?
   • Does each muon get placed in the same TME? ( I suspect yes).
   • None of these muons should be used for physics, but we can still produce TMEs from them, to check pile-up or detector responses etc.
   • When do we consider the timestamp of the TME to be?

Actually, thinking more about it, perhaps the timestamp of a TME in a global sense is not meaningful. I guess we's only care about the relative times within the TME.

[jrquirk]

not overlap with any other TME.

The original idea was that this is not the case. Each muon gets to be the center of its own TME, thus answering a question or two you have later. That means a muon might be in another's window, and it will be the center of one TME and off-center in another.

• None of these muons should be used for physics, but we can still produce TMEs from them, to check pile-up or detector responses etc.

We will use this for certain physics. We wouldn't use them, for instance, for lifetime measurements. But we can get a total-number-of-something/total-number-of-muons measurement from them still. Additionally, there's even still physics outside TMEs that we'll be analysing with TDPs or something TME-like, for instance delayed gammas from long-lived capture daughters (Mg-27, for instance, which we can measure far away from a muon stop for a clean sample) or just background.

[benkrikler]

not overlap with any other TME.

The original idea was that this is not the case. Each muon gets to be the center of its own TME, thus answering a question or two you have later.

Oh, I see. So they can overlap, but we're not worried about over-counting as they'll all be flagged for pile-up so handled delicately in subsequent analysis. That does answer my other questions then.

for instance delayed gammas from long-lived capture daughters (Mg-27, for instance, which we can measure far away from a muon stop for a clean sample) or just background.

That's a good point. Does this suggest TMEs can have no muons? One of the key goals of the TME is to make later analysis easier whilst looking at correlation between detectors, right? If we'd want to say give me TDPs from a detector when nothing else saw something, then perhaps the TME container would make this easier. I can also see how I would implement this within the algorithm producing proper TMEs.

One other thing that we've not mentioned much. All of the TAPs that form TDPs which then get grouped together to form a TME will have had their times corrected by the TDiff values compared to the MuSc (at least, I've always been assuming that). This means that whilst the muon itself could arrive squarely in the middle of a midas event, the protons that we then see at the silicon could easily be near the edge of the event. Do we need to account for this sort of issue within the TME itself? Does each TDP need to be weighted or carry such information with it once it's put into the TME?

[benkrikler]

I'm starting to re-implement all of the TME stuff as described above. Two questions remain:

1. Do we construct TMEs from TAPs or TDPs?
2. How do we handle pulses that were near the beginning / end of a midas event? We say we would ignore muons within a certain window of the edge of a midas event but what about other channels? The timing offsets we apply will mean one channel could be at the edge of a midas event, whilst the others aren't.

For question 1, I'm going to start with TDPs but will bear in mind that this may want changing in the future.

[AndrewEdmonds11]

For question 1, I'm going to start with TDPs but will bear in mind that this may want changing in the future.

I think this is fine but why would we want to change it? Also, do we create muSc TDPs even though there is only one channel?

How do we handle pulses that were near the beginning / end of a midas event?

Is this not something we consider and deal with after creating the TMEs? In which case, we don't need to worry about this too much now but we should bear it in mind for later.

[benkrikler]

Actually, question 2 above turns out to be a bigger issue than I originally thought. Our main event loop only provides access to a single MIDAS event for all detectors. Changing this would need a lot of work to rewrite the main event loop and change how other modules expect to receive pulses.

Fortunately, this may only be a problem if the TDiffs we need to apply for each detector turn out to be very long compared to the length of a MIDAS event. I'll continue implementing this for now as if it weren't a problem, but we'll need to see what efficiencies we get for matching pulses once improved TDiff values have been found.

The same problem could affect the [creation of] TDPs, although since we only pair 2 channels we should be less affected.

[jrquirk]

As long as a muon has the +/- window_time on either side of it, pulses in there should be good. We just need to be able to see into the veto window, and in this context we've only talked about not using muons within +/-10us of the edges. Other pulses don't have this restriction because we're not using them as vetos. If at any point in the analysis we decide to say something like "we won't count a silicon hit if there is a scintillator veto hit within 5us of it", then we'd have to reexamine this.

I don't understand why the TDiffs are an issue. TAPs should already have them applied. The longest ones are on the order of tens of microseconds.

Does the TGlobalData have any idea of timing with respect to the ends? The beginning is doable possibly because we're counting up clock ticks from zero. But what about the other end?

[benkrikler]

"we won't count a silicon hit if there is a scintillator veto hit within 5us of it"

I thought that this sort of task was one of the main reasons to go to a muon centred tree?

I don't understand why the TDiffs are an issue. TAPs should already have them applied. The longest ones are on the order of tens of microseconds

It's not an issue of applying the TDiffs to TAPs, it's an issue that one TME could actually overlap more than one Midas event, which we don't have access to. Midas blocks are 110 microseconds long, right, so TDiffs of tens of microseconds make the chance of missing pulses in one channel when building the TME is not insignificant.

Does the TGlobalData have any idea of timing with respect to the ends?

That's a good point, but I'm not sure. I don't remember us ever looking at that information before. Do the original Midas files at least contain the total time for each MIDAS event?

[jrquirk]

I think I obfuscated the issue by trying to provide an example. I meant an ScR or ScL hit, but I didn't specify. But yeah, the muSc hit is why we're doing it this way. I just meant that for now, since all but the pile-up cuts are prompt, we just need to ensure that +/-10us of a muon are within our measurement window. Any pulses that occur here I'm pretty sure are okay.

The MIDAS blocks are 110 ms, not 110 us. There's no way, even if the MIDAS blocks were close, to correlate events from one to the other. We didn't measure the dead time. I don't know how much information we have on the length of a MIDAS event. We may have to use the first and last hits as bookends.

The issue of the ends of the measurement window getting shifted is still an issue, though. Because if we have something like

muSc:++++++++++|-----------------| SiR2:+++++++++++|------------------| SiR1:++++++++++++|------------------|

(We don't have a perfectly synchronous start/stop I imagine) And then we shift the pulses, we've shifted the measurement window

muSc:++++++++++|-----------------| SiR2:++++++++|------------------| SiR1:+++++++++++++++|------------------|

So where's the start and end of the measurement window? I think this is a good Wednesday discussion topic.

Edit: The + don't represent anything, I just used it for spacing because whitespace disappears. The - is the measurement window.

[AndrewEdmonds11]

The diagrams looked good in my e-mail:

muSc:          |-----------------|
SiR2:           |------------------|
SiR1:            |------------------|

and

muSc:          |-----------------|
SiR2:        |------------------|
SiR1:                |------------------|

[benkrikler]

Ah, good so that's much less of an issue if we have 110ms events not 110us (bit of a brain-fart there, of course it's not 110us).

Your ascii art shows what I've been trying to describe perfectly. From your last one:

muSc:    |----+===========+--|
SiR2:  |------+===========|
SiR1:         |===========+------|

we can only be confident of catching all of a TME if the window around the central muon falls completely within the high-lighted (by equals signs) region.

So from what I gather:

• How likely are we to TMEs that fall outside of this? - Not likely if the max TDiff O(10us) compared to 110ms gates.
• Handling early TMEs: We can only identify this problem with certainty for TMEs that occur at the beginning of an event, for those we can flag them explicitly.
• Handling late TMEs: All TMEs that look at a channel which has no more pulses for this midas event may suffer this problem, but we can't be sure since we only know the length of the event approximately. We can still flag TMEs whose window is within some buffer region of the end of an event (defined by the gate length uncertainty which we can get from the LLDQ timestamp plots perhaps) and who don't have any more pulses after the window.

[litchfld]

How do we handle pulses that were near the beginning / end of a midas event? We say we would ignore muons within a certain window of the edge of a midas event but what about other channels? The timing offsets we apply will mean one channel could be at the edge of a midas event, whilst the others aren't.

Normally you set your equipment up so the answer is 'junk it'. I think that is fine, if i've read correctly the dead fraction induced is: max(TDiff)+window / window ~ 10us+10us /100ms ~ 0.5%, which is tiny.

A related question is propagation/ reaction delay for the each digitiser to start outputting when it is told to start reading out. Sorry, this is notoriously hard to visualise (and the plenary session i'm in is about to end) but the basic point is that the TDiff generally comes from the delay of clock synchronisation signals (the earlier timestamped detectors have slower clock distribution) while the deadtime is determined by the delay of 'start readout' signals.

I think the clock synchronisation is derived from the 'start event' signal in our case? In which case there should not be much difference, just the analogue upstream delays but the signals have to separate somewhere and I don't know for sure the relative latency is small.

Hopefully these delays are stable, at least between re-cablings, so we can just use a large ensemble of timestamps (noise is fine!) to measure this empirically.

[jrquirk]

With regards to knowing when things end, Jim suggested looking at the timestamp distribution and determining when it's no longer flat. We can also predict when the end of a MIDAS event should be (110 ms / clock_tick_in_ms) and use a large buffer window from there. This can all be applied after making the TMEs I think, as long as there is still some access to the lower-level information in some way.

[benkrikler]

That sounds very similar to what I described above; I think it's the best we can do.

I've put in some extra flags to the TME definition and have been writing things in the expectation that the TME generator would fill this, though we should be able to cross check this in later analysis by other means. Certainly I think TMEs contain all the necessary information.

I've opened issue #200 to cover the task of producing the length and uncertainty of a MIDAS event.
In the mean time I'll finish the TME work with some coarse estimates.