Rate Significance Calculator on Output Tab- upgrade

[stdye]

Two restrictive simplifications make the "calculator" less useful than it could be.

1) The significance calculation considers only statistical uncertainty; It ignores systematic uncertainty on the background.

Because the antineutrino backgrounds are estimated, the significance of detecting the signal is degraded by the systematic uncertainty of these estimates. A better expression for the significance is

(Written originally in this comment as

N_sigma = Signal x sqrt(Time) / sqrt(Signal + Background + deltaBackground^2),

but see comments below for corrected equation)

where deltaBackground^2 is the sum of the squares of the systematic uncertainties of the rates of the various backgrounds.

Default values of the fractional systematic uncertainties of the rates would be 25% for geo-nus and 6% for reactor antinus. Users should have the ability to set the quantities to different values.

2) Results apply only to a detector with 10^32 targets.

Users should enter the number of targets as a factor of 10^32. The significance (N_sigma) is then

N_sigma = Signal * sqrt(Time x Factor) / sqrt(Signal + Background + deltaBackground^2).

In the meanwhile, a note that tells users to account for this factor needs to be added to the Rate Significance Calculator Pane.

Note: Coding up a new variable for the exposure factor can be avoided if the "Solve For" option is simply "Exposure" rather than "Exposure Time". In this case the value-setting box now called "Exposure Time for 10^32 Targets" with unit "years" becomes simply "Exposure" with unit "10^32 target-years". Users can easily solve for the time using the number of targets for their detector.

[stdye]

CAUTION:

N_sigma = Signal_Rate x sqrt(Time) / sqrt(Signal_Rate + Background_Rate + deltaBackground_Rate^2)

is NOT correct. Squaring the uncertainty in the background rate gives it different dimensions than the signal and background rates, which are not squared. Mea culpa!

The following equation is correct

N_sigma = S / sqrt(S + B + deltaB^2),

where S is the NUMBER, not the rate, of signal events and B is the NUMBER, not the rate, of background events,.

S = Signal (NIU) x Factor (number of 10^32 targets) x Time (yr)

B = Background (NIU) x Factor (number of 10^32 targets) x Time (yr)

deltaB = B x deltaB/B,

with default values of (deltaB/B)_geonu = 0.25 and (deltaB/B)_reactor_antinu = 0.06

[DocOtak]

Currently trying to grok this one for implementation...

I did a little test just solving for significance: Nσ = (Signal Time) / sqrt(Signal + Background Time)

This results in the significance rising faster than the current equation (as a function of exposure time)

For the uncertainties: detlaB^2 = ((backgound_genu 0.25 )^2+ (background_cores 0.06)^2)

where the 0.25 and 0.06 are user set-able when that is ready.

Does the above sound about right?

[stdye]

Not quite right. Adding systematic uncertainty should always decrease the significance. Note that the current equation includes only the statistical uncertainty.

The following equation is correct:

Nσ = Signal_Rate Exposure / sqrt[(Signal_Rate+Background_Rate) Exposure + (Uncertainty_Background_Rate * Exposure)^2]

where for rates in NIU, Exposure is the product of time in years and the number of 10^32 targets in the detector.

Solving this equation for Exposure gives:

Exposure = [(Nσ)^2 (Signal_Rate + Background_Rate)] / [Signal_Rate^2 - (Nσ)^2 (Uncertainty_Background_Rate)^2]

In the absence of systematic uncertainty in the Background_Rate, exposure increases with the square of the significance (eg. Doubling the significance requires a 4-fold increase in exposure; This is just Poisson or counting statistics). Including systematic uncertainty in the Background_Rate increases the exposure required to achieve a given significance. This is indicated by the minus sign in the denominator.

The Rate Significance Calculator drop-down menu for "Signal (background)" has 6 options for the various signal and background combinations. Explicitly, the following gives the expressions for the background uncertainty for several options.

1) Signal_Rate = All cores; Background_Rate = Geonu. In this case Uncertainty_Background_Rate = Geonu_Rate 0.25 2) Signal_Rate = Closest_Core; Background_Rate = Geonu + other cores. In this case Uncertainty_Background_Rate = sqrt[(Geonu_Rate 0.25)^2 + (Other_cores_Rate 0.06)^2] 3) Signal_Rate = Custom_Core; Background_Rate = Geonu + other cores. In this case Uncertainty_Background_Rate = sqrt[(Geonu_Rate 0.25)^2 + (Other_cores_Rate 0.06)^2] 4) Signal_Rate = Geonu; Background_Rate = Reactor cores. In this case Uncertainty_Background_Rate = Reactor_Rate 0.06

[stdye]

The previous comment on this thread has been edited; The * symbol, which is used to indicate multiplication, typed without a space after it makes italicized text

[stdye]

When coding for the equation solved for Exposure above, the condition:

(Nσ) * (Uncertainty_Background_Rate) > (or =) Signal_Rate

must be avoided. The equal sign gives the upper limit (infinite exposure) to the significance of detecting the signal in the presence of the background with systematic uncertainty; Nσ < Signal_Rate / Uncertainty_Background_Rate

[stdye]

In addition to the background from geo-neutrinos and reactor antineutrinos, detectors inevitably record background from various non-neutrino sources (i.e. fast neutrons and spallation products from cosmic rays, unavoidable radioactivity in the detector, etc.). This could be called nuisance background.

The Significance Calculator could allow users to add the rate of nuisance background and the fractional systematic uncertainty associated with it.

Including nuisance background in the significance calculation is straightforward. Simply add the rate to the rates from the other background sources, while the uncertainty in the nuisance rate gets added in quadrature with the other background rate uncertainties.

Uncertainty_Background_Rate = sqrt(Uncertainty_Geonu_Background_Rate^2 + Uncertainty_Reactor_Background_Rate^2 + Uncertainty_Nuisance_Background_Rate^2).

Users would find it useful to know the infinite exposure upper limit to the Uncertainty_Nuisance_Background_Rate for a given significance level. The calculator could easily provide this limit.

Note: One outstanding issue to be resolved is the spectral shape of nuisance background.

[stdye]

This is a partial list of things to do to further upgrade the Significance/Exposure Calculator.

1) Limit exposures to those yielding 2 or more signal + background "events". Who would believe an experiment that recorded 1 event? In fact, given the analytical nature of the calculator, the significance/exposure results could be based on fractional event totals, even those less than 1. The minimum number of events might need to be a user-settable value but certainly no less than 2.

2) For nuclear reactor monitoring demonstrations, there is much current interest in estimating the exposure required to detect all cores within a given complex rather than the closest core (e.g. If/when the Hartlepool cores permanently shut down, what are the prospects of detecting the cores at the Heysham complex?). Maybe users could select the complex?

3) Allow users to add nuisance background and its fractional systematic uncertainty. Initially assume that the spectrum is flat in energy. In other words, the Antineutrino E_min and E_max sliders have a linear effect on nuisance background.

4) Add a space between "10^32" and "target-years" in the unit displayed on the Exposure value-setting box.

[stdye]

PR attempts to address 1) and 3) above. See what you think.