Questions about CLs_output_summary.dat

[Revue-Starlight-Topstar]

Question

Hi @jackaraz After getting CLs_output_summary.dat in the reconstructed-level mode, I found a strange thing: CLs_output_summary.dat.txt A signal region [SL] - SRS A that does not exist in cms_sus_16_039.info appears on line 173. Its appearance makes the best signal region of cms_sus_16_039 not unique. And it has no corresponding value of efficiency and stat. Does [SL] - SRS A have any special meaning? Many thanks.

[jackaraz]

Hi @Revue-Starlight-Topstar, that is the global confidence level constructed by using a covariance matrix for the background channels within a simplified likelihood. Correlates all the signal regions presented in that analysis and provides more accurate CLs calculation. A preliminary study has been presented in 2002.12220 and we will release the proper validation paper this month for that feature.

[Revue-Starlight-Topstar]

Hi @jackaraz Can I think that if the 1-CLs value of the line where [SL] - SRS A is greater than or equal to 0.95, it means that the input model points are excluded?

[jackaraz]

Hi @jackaraz Can I think that if the 1-CLs value of the line where [SL] - SRS A is greater than or equal to 0.95, it means that the input model points are excluded?

Yes, the way the results are presented is the same; if there is no global likelihood information provided, you need to find the best SR (see this paper for details) and check its 1-CLs value. Assuming 95% CL for the exclusion limit, then 1-CLs value > 0.95 means that your model with the given parameter space has been excluded. The same goes for the global one, which overwrites the individual exclusion limits.

[Revue-Starlight-Topstar]

@jackaraz Oh, I got it. Thank you very much.

[Revue-Starlight-Topstar]

Hi @jackaraz It suddenly occurred to me that after each submit, there will be the following prompt:

MA5: Would you like to edit the recasting Card ? (Y/N)
Answer: n

It's troublesome to answer every time. Is there a way to turn it off? Many thanks.

[jackaraz]

You can use the script mode or forced mode.

[Revue-Starlight-Topstar]

Hi @jackaraz I still have some unclear places. (1)About the best SR：If the value of 1-cls in the best signal region is less than 0.95 and there is a value of 1-cls greater than 0.95 in other signal regions, is the model with a given parameter space allowed or excluded?What is the special significance of the best SR? (2)In my example, [SL] - SRS A represents the global confidence level of cms_sus_16_039 or the global confidence level of all analyses used? If I want to determine whether the model is allowed or excluded, just look at the line where [SL] - SRS A is located? Or do I have to find the value of 1-cls of the best signal region for each analysis and check whether there is a value greater than or equal to 0.95? Many thanks.

[jackaraz]

Hi @Revue-Starlight-Topstar

(1)About the best SR：If the value of 1-cls in the best signal region is less than 0.95 and there is a value of 1-cls greater than 0.95 in other signal regions, is the model with a given parameter space allowed or excluded?What is the special significance of the best SR?

The best signal region signifies the most sensitive SR, assuming that you have a statistically valid sample. If you lack statistics the best SR can change widely so it is quite important to generate a large enough sample.

(2)In my example, [SL] - SRS A represents the global confidence level of cms_sus_16_039 or the global confidence level of all analyses used? If I want to determine whether the model is allowed or excluded, just look at the line where [SL] - SRS A is located? Or do I have to find the value of 1-cls of the best signal region for each analysis and check whether there is a value greater than or equal to 0.95?

This depends on how experimental collaboration released the information. A simplified/full likelihood can represent all or some of the SRs presented in the analysis. For details, I suggest reading the paper from the experimental collaboration or waiting for our paper which should be released in a couple of weeks. We will be releasing details about all available analyses which include this global likelihood information.

[Revue-Starlight-Topstar]

Hi @jackaraz (1) As long as sufficient statistical data are available, the 1-CLs value of best SR must theoretically be the largest of all SRs. Do I understand that correctly? (2)What is the way to determine if a signal region is the best SR? Many thanks.

[jackaraz]

Hi @Revue-Starlight-Topstar

(1) As long as sufficient statistical data are available, the 1-CLs value of best SR must theoretically be the largest of all SRs. Do I understand that correctly?

Not necessarily. Best SR shows you the most sensitive SR for your signal phase-space. It does not guarantee the relative value of CLs.

(2)What is the way to determine if a signal region is the best SR?

The best signal region is the one with the maximum ratio of the number of signal events per expected number of events at 95% CL.

[Revue-Starlight-Topstar]

Hi @jackaraz The luminosity of analysis atlas-susy-2018-32 is 139fb-1.I would like to briefly discuss the exclusion limits when the luminosity is 300fb-1. Directly apply set main.recast.add extrapolated_ luminosity \=300, Madanalysis5 will by default automatically change the number of background events to the value corresponding at luminosity of 300fb-1? Many thanks.

[jackaraz]

Hi @Revue-Starlight-Topstar set main.recast.add.extrapolated_luminosity = 300 3000 will automatically scale everything and report the CLs values accordingly. You need to be careful about main.recast.error_extrapolation, where you need to assume that the error is either statistical or systematically dominated. Please see fig 5 in this paper and the discussion around it.

PS there is no \ that is to indicate the continuing line in the paper.