search

alcap-org / AlcapDAQ

Alcap DAQ
alcap-org.github.io
8 stars 0 forks source link

Switch 2D histograms to use THnSparse #193

Open benkrikler opened 10 years ago

benkrikler commented 10 years ago

I've just come across the ROOT histogram class, THnSparse. It's designed specifically to handle histograms where most bins are empty which would hugely reduce the amount of memory the histogram consumes.

Most of the bad-allocs I've seen were due to too many histograms with too many bins. As a result many of our plots have had there binning reduced, which white-washes many of the small speaks. With this sparse histogram version, we'd be able to keep smaller bin widths and consume far less memory.

Does anyone have experience with this histogram type?

I'm not sure whether this should be a priority though if we want to increase binning immediately or whether we should focus on other things now. I'm putting it to the long term milestone for now.

AndrewEdmonds11 commented 10 years ago

How much work would it be to change? Is it just a case of doing a search and replace of TH2F to THnSparse?

benkrikler commented 10 years ago

THnSparse provides a single interface for an arbitrary number of dimensions. That means the constructor and the Fill method are different to normal TH1 / TH2s in that they use arrays to pass in values. Each array should have a length equal to the number of dimensions in the histogram. So this is not quite as simple as search and replace, although probably not too much hard work.

litchfld commented 10 years ago

This is getting kind of weird... Its not normal to run out of memory allocating histograms. I can't help feeling something is more fundamentally wrong in what we are doing.

I've not used this class, but the basic concept is really intended for high dimension (>4) histograms, which do become sparse very quickly. So our 2D histograms may not be sparse enough to benefit (I guess you need to be lower than about 10% filled so see anything significant). And if there is a significant effect, you likely either have a bad range or too fine binning.