Suspicious number of hits

[airqui]

Hi @artlbv, applying the build_events I found out that the message "suspicious number of hits" appears quite often if len(hits)>10000 I am not sure to understand what this 10000 means. It is impossible to have 10000 hits, where does it comes from?

[airqui]

Most of them happen in bcid 3097-ish:

Suspicious number of hits! 67009 for bcid 3097 Skipping event 85278097 Entry 19900 Entry 56500 Entry 17100 Entry 20000 Suspicious number of hits! 98433 for bcid 3097 Skipping event 92533097 Suspicious number of hits! 102337 for bcid 3097 Skipping event 100333097 Suspicious number of hits! 104516 for bcid 3097 Skipping event 282828097 Suspicious number of hits! 101121 for bcid 3097 Skipping event 56653097 Entry 56600 Entry 20100 Entry 17200 Suspicious number of hits! 57729 for bcid 3097 Skipping event 56718097 Suspicious number of hits! 52801 for bcid 3097 Skipping event 92668097 Entry 56700 Suspicious number of hits! 103041 for bcid 3097 Skipping event 100363097 Entry 56800 Entry 20100 Suspicious number of hits! 68742 for bcid 3097 Skipping event 92848097 Suspicious number of hits! 52168 for bcid 3097 Skipping event 56898097 Suspicious number of hits! 87106 for bcid 3096 Skipping event 86243096

[artlbv]

In theory, there cannot be more than 7*1024, 7168, hits in a single event/BCID. In practice, this is also very very unlikely. Therefore I've limited the hit storage array to a length of 10k. It seems that due to the BCID merging there is this particular BCID ~3097 which collects many hits. Since it's systematic, I have skipped it for now, but I plan to look into it in more detail to understand the origin of this strange behaviour.

[airqui]

@artlbv I was looking at this. The issue appears at the very first level, before any merging. You can see from the figures here that some of the slabs have bcid noise burst in ~2900 and then some funny structures after. One of them in ~3090.

But, I also have produced, using the output of the merged file, a histogram of nhits in bcid=3097 and it looks okay. So the problem has to be in the merging of bcid+-1. It seems that the script is merging bad events with good ones.

Could you look at this ? It might be that the bcid+-1 merging is not properly done and we may introduce a lot of noise.

[airqui]

PS: the last figure is produced for only few events, but the build_events.py complains already before 100 events.

[airqui]

PS2: A bad new is that the runs that I am suing are MIP runs, so the bcid noise burst issue appeared at the beginning of the testbeam.

[airqui]

One extra piece of information: Analyzing the file, MIPscan/rootfiles_bcidTh15/fullscan/3GeV_calibration_merge.root

I have added some couts and see that in the entry 79, we have a suspicious event without having done any bcid merge (we should see "Found nearby" etc just after "entry 79 etc"

entry 79, event num 395248 n hits = 1, bcid = 1346 n hits = 1, bcid = 2042 n hits = 1, bcid = 2234 n hits = 1, bcid = 2343 n hits = 1, bcid = 2855 n hits = 104897, bcid = 3097 Suspicious number of hits! 104897 for bcid 3097 and previous bcid 2855

If we look in the root file there are no more than 1-3 hits per chip for all entries < 81 if badbcid<2

[artlbv]

actually, I was using a bcid +/- 2 window, but that does not help. it seems this 3097 has no real hits, and the HG/LG are -999. so there's something deeper... I'm trying to figure out

[artlbv]

Btw, interestingly, this appeared only after I've introduced the bcid correction: https://github.com/SiWECAL-TestBeam/SiWECAL-TB-analysis/commit/c2a8e1bcbaeeb23457e9c8101560a6ee20f9d9e4 if I return just the old bcid, there are no such problems in 3096. bizarre

[airqui]

Modifying the get_hit funtion a bit, like bellow, the message dissapears. It is correct?

The badbcid variable is chip by chip based, I think that in the "get_good_bcids" we do this also chip by chip, but then in the get_hits, we do it more global. So if the 3097 was okay for one chip, we calculate the number of hits for all chips and maybe for them was not correct.

badbcid = entry.badbcid

for slab in xrange(NSLAB):
    for chip in xrange(NCHIP):
        for sca in xrange(NSCA):

            sca_indx = (slab * NCHIP + chip) * NSCA + sca
            bcid = get_corr_bcid(entry_bcids[sca_indx])

            if badbcid > 1 or badbcid < 0: continue

            # filter bad bcids    

corrected: I think that this is not correct...

[airqui]

seems something deep :-( ... I go home ! I am hungry.

[artlbv]

this part: https://github.com/SiWECAL-TestBeam/SiWECAL-TB-analysis/blob/master/build_events.py#L75

                # filter bad bcids
                if bcid not in bcids: continue
                # filter merged bcids
                #if bcids[bcid] == 0: continue

                ## if merged bcid, find closeby bcid
                if bcids[bcid] == 0:
                    for bcid_close in [bcid-1,bcid+1,bcid-2,bcid+2]:
                        if bcid_close in bcids:
                            if bcids[bcid_close] > 0: bcid = bcid_close

should check whether the current bcid is in the "good bcids" list, and then checks whether it's merged. but it seems not to work well

[airqui]

A question: it is possible that the bcid is in goodbcids for chip 15 but not for another one and we construct the number of hits considering also retriggers happening at the same bcid in other chip? If this is the case (I am still not sure how it works the script) we might want to tag or remove these events ... not only the ones with > 10000 hits, but all that events were the badbcid>2 for some chips but not for the others. And we also see if this happens in early bcid's. If we are lucky, this issue will not afffect the early bcids.

[artlbv]

That's a good point! For now I was just looping over the list of BCIDs and checking the flag for each chip basically. Instead, one would need to maybe fill the list of bcids and also a flag whether it's good directly. Then, one would just reject the bcid if it was bad for any chip. That should be easy to implement.

Le 5 août 2017 à 14:30, airqui notifications@github.com a écrit :

A question: it is possible that the bcid is in goodbcids for chip 15 but not for another one and we construct the number of hits considering also retriggers happening at the same bcid in other chip? If this is the case (I am still not sure how it works the script) we might want to tag or remove these events ... not only the ones with > 10000 hits, but all that events were the badbcid>2 for some chips but not for the others. And we also see if this happens in early bcid's. If we are lucky, this issue will not afffect the early bcids.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[airqui]

Can you do this? Honestly, I still don't fully understand how indexes work in this script... Despite this, I start to like python

[artlbv]

Ok, I've got it. It's a very stupid bug :D When I've introduced the get_corr_bcid function which corrects the bcid for the overloop, I've ignored that the empty bcid is -999 and since it's < 1245, I've added 4096 -999 + 4096 = 3097. Voila!

[airqui]

Great! What about what we discussed here https://github.com/SiWECAL-TestBeam/SiWECAL-TB-analysis/issues/7#issuecomment-320441149 ?

[artlbv]

It's here: #9 , but I did not notice any significant changes for now.