Numpy version dependency

[APMDSLHC]

System Settings

Fedora Linux 35 Python 3.9.12

Describe the bug

numpy warnings library is not available anymore in numpy version 1.24.1 ?

In order to load a statistical model, Madstats needs to import PyhfInterface from madstats.backends.pyhf_backend.interface, which needs to import warnings from numpy.

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Input In [1], in <cell line: 13>()
     10 with open("./madstats/test/background_test.json", "r") as f:
     11     background = json.load(f)
---> 13 stat_model = madstats.get_multi_region_statistical_model(
     14     analysis="atlas_susy_2018_31",
     15     signal=signal,
     16     background=background,
     17     xsection=0.000207244,
     18 )
     19 print(stat_model)
     20 # Out: StatisticalModel(analysis='atlas_susy_2018_31', xsection=2.072e-04 [pb], backend=pyhf)

File ~/SModelS/madstats/src/madstats/__init__.py:105, in get_multi_region_statistical_model(analysis, signal, background, covariance, nb, third_moment, delta_sys, xsection)
    102 assert len(signal) > 1, "Incorrect input shape."
    104 if isinstance(signal, list) and isinstance(signal[0], dict) and isinstance(background, dict):
--> 105     from madstats.backends.pyhf_backend.interface import PyhfInterface
    106     from madstats.backends.pyhf_backend.data import Data
    108     model = Data(signal=signal, background=background)

File ~/SModelS/madstats/src/madstats/backends/pyhf_backend/interface.py:3, in <module>
      1 import logging, scipy
      2 from typing import Dict, Union, Optional, Tuple
----> 3 from numpy import warnings, isnan
      4 import numpy as np
      6 import pyhf

ImportError: cannot import name 'warnings' from 'numpy' (/home/pascal/anaconda3/lib/python3.9/site-packages/numpy/__init__.py)

To Reproduce

import madstats, json
import numpy as np

with open("./madstats/test/signal_test.json", "r") as f:
    signal = json.load(f)

with open("./madstats/test/background_test.json", "r") as f:
    background = json.load(f)

stat_model = madstats.get_multi_region_statistical_model(
    analysis="atlas_susy_2018_31",
    signal=signal,
    background=background,
    xsection=0.000207244,
)

Expected behaviour

The code should load the warning library of numpy.

Additional information

The bug appears with numpy version 1.24.1 but not with version 1.21.6. It may or may not work for in-between versions.

[jackaraz]

Hi @APMDSLHC, this should now be eliminated, assuming that I didn't miss any.

[APMDSLHC]

Thanks. I now have another dependency issue with numpy, which has not the attribute float. I use numpy version 1.24.1.

Here is the full error message, even though only the last line is probably needed to understand the issue.

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Input In [3], in <cell line: 21>()
     18 print(sl_model)
     19 # Out: StatisticalModel(analysis='cms_sus_19_006', xsection=2.072e-04 [pb], backend=simplified_likelihoods)
---> 21 print(f"1 - CLs : {sl_model.exclusion_confidence_level()}")
     22 # Out: 1 - CLs : 0.40245643517834495
     24 print(f"Expected exclusion cross-section at 95% CLs : {sl_model.s95exp}")

File ~/SModelS/madstats/src/madstats/interface/statistical_model.py:114, in StatisticalModel.exclusion_confidence_level(self, expected, **kwargs)
    107 if self.backend_type is AvailableBackends.pyhf:
    108     kwargs.update(
    109         dict(
    110             CLs_obs=expected in [ExpectationType.apriori, ExpectationType.observed],
    111             CLs_exp=expected == ExpectationType.aposteriori,
    112         )
    113     )
--> 114 return self.backend.computeCLs(mu=1.0, expected=expected, **kwargs)

File ~/SModelS/madstats/src/madstats/backends/simplifiedlikelihood_backend/interface.py:243, in SimplifiedLikelihoodInterface.computeCLs(self, mu, expected, marginalise, allow_negative_signal, iteration_threshold)
    225 """
    226 Compute 1 - CLs value
    227 
   (...)
    234 :return: 1 - CLs
    235 """
    236 min_nll_asimov, negloglikelihood_asimov, min_nll, negloglikelihood = self._exclusion_tools(
    237     expected=expected,
    238     marginalise=marginalise,
    239     allow_negative_signal=allow_negative_signal,
    240     iteration_threshold=iteration_threshold,
    241 )
--> 243 return 1.0 - compute_confidence_level(
    244     mu, negloglikelihood_asimov, min_nll_asimov, negloglikelihood, min_nll
    245 )

File ~/SModelS/madstats/src/madstats/tools/utils_cls.py:26, in compute_confidence_level(mu, negloglikelihood_asimov, min_negloglikelihood_asimov, negloglikelihood, min_negloglikelihood)
     14 """
     15 Compute confidence limit with respect to a given mu
     16 
   (...)
     23 :return: confidence limit
     24 """
     25 if isinstance(mu, (float, int)):
---> 26     mu = np.array([mu]).astype(np.float)
     27 elif len(mu) == 0:
     28     mu = np.array([mu])

File ~/anaconda3/lib/python3.9/site-packages/numpy/__init__.py:284, in __getattr__(attr)
    281     from .testing import Tester
    282     return Tester
--> 284 raise AttributeError("module {!r} has no attribute "
    285                      "{!r}".format(__name__, attr))

AttributeError: module 'numpy' has no attribute 'float'

[jackaraz]

Thanks @APMDSLHC this should be fixed now.

[APMDSLHC]

Ok, I don't know if it's related to previous issues but now I have this error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [1], in <cell line: 13>()
     10 with open("./madstats/test/background_test.json", "r") as f:
     11     background = json.load(f)
---> 13 stat_model = madstats.get_multi_region_statistical_model(
     14     analysis="atlas_susy_2018_31",
     15     signal=signal,
     16     background=background,
     17     xsection=0.000207244,
     18 )
     19 print(stat_model)
     20 # Out: StatisticalModel(analysis='atlas_susy_2018_31', xsection=2.072e-04 [pb], backend=pyhf)

File ~/SModelS/madstats/src/madstats/__init__.py:106, in get_multi_region_statistical_model(analysis, signal, background, covariance, nb, third_moment, delta_sys, xsection)
    103 assert len(signal) > 1, "Incorrect input shape."
    105 if isinstance(signal, list) and isinstance(signal[0], dict) and isinstance(background, dict):
--> 106     from madstats.backends.pyhf_backend.interface import PyhfInterface
    107     from madstats.backends.pyhf_backend.data import Data
    109     model = Data(signal=signal, background=background)

File ~/SModelS/madstats/src/madstats/backends/pyhf_backend/interface.py:19, in <module>
     15 pyhf.workspace.log.setLevel(logging.CRITICAL)
     16 pyhf.set_backend("numpy", precision="64b")
---> 19 class PyhfInterface(BackendBase):
     20     """
     21     Pyhf Interface
     22 
     23     :param model: contains all the information regarding the regions, yields
     24     :raises AssertionError: if the input type is wrong.
     25     """
     27     __slots__ = ["_model", ]

File ~/SModelS/madstats/src/madstats/backends/pyhf_backend/interface.py:161, in PyhfInterface()
    147         return CLs["CLs_obs"]
    149     return CLs
    151 def likelihood(
    152     self,
    153     mu: Optional[float] = 1.0,
    154     expected: Optional[ExpectationType] = ExpectationType.observed,
    155     allow_negative_signal: bool = True,
    156     return_nll: Optional[bool] = False,
    157     return_theta: Optional[bool] = False,
    158     isAsimov: Optional[bool] = False,
    159     iteration_threshold: Optional[int] = 10,
    160     options: Optional[Dict] = None,
--> 161 ) -> Union[float, List[float, np.ndarray]]:
    162     """
    163     Compute the likelihood of the given statistical model
    164 
   (...)
    196     :return: (float) likelihood
    197     """
    199     _, model, data = self.model(mu=1.0, expected=expected)

File ~/anaconda3/lib/python3.9/typing.py:277, in _tp_cache.<locals>.decorator.<locals>.inner(*args, **kwds)
    275 except TypeError:
    276     pass  # All real errors (not unhashable args) are raised below.
--> 277 return func(*args, **kwds)

File ~/anaconda3/lib/python3.9/typing.py:838, in _SpecialGenericAlias.__getitem__(self, params)
    836 msg = "Parameters to generic types must be types."
    837 params = tuple(_type_check(p, msg) for p in params)
--> 838 _check_generic(self, params, self._nparams)
    839 return self.copy_with(params)

File ~/anaconda3/lib/python3.9/typing.py:214, in _check_generic(cls, parameters, elen)
    212 alen = len(parameters)
    213 if alen != elen:
--> 214     raise TypeError(f"Too {'many' if alen > elen else 'few'} parameters for {cls};"
    215                     f" actual {alen}, expected {elen}")

TypeError: Too many parameters for typing.List; actual 2, expected 1

[jackaraz]

That's not related to NumPy, but it's fixed now.

[APMDSLHC]

Ok thanks, everything seems to work fine now.

[jackaraz]

Great closing the issue then, thanks!