Support single operator constrain with `single_parameter_fit: True`

[ecelada]

I tried to run an individual fit just adding to the runcard single_parameter_fit: True and then as usual "smefit NS runcard.yaml". However I get the following error

Traceback (most recent call last): File "/Users/eugeniacelada/Library/Python/3.10/bin/smefit", line 8, in sys.exit(base_command()) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/click/core.py", line 1130, in call return self.main(args, kwargs) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, ctx.params) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/click/core.py", line 760, in invoke return __callback(args, **kwargs) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/smefit/cli/init.py", line 79, in nested_sampling runner.run_analysis("NS") File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/smefit/runner.py", line 215, in run_analysis self.single_parameter_analysis(optimizer) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/smefit/runner.py", line 180, in single_parameter_analysis self.ns(single_coeff_config) AttributeError: 'Runner' object has no attribute 'ns'

[giacomomagni]

Hi @ecelada, I'm not able to reproduce your error straight. Can you add here the runcard??

Keep in mind that in the current master the previous implementation of Nested Sampling has changed, and the attibute .ns has indeed been removed.

But if I run with smefit NS path_to_my_runcard with single_parameter_fit: True I don't get any error.

[ecelada]

Hi, I copy here my card, could you let me know if you spot something wrong? Thank you

# Input YAML configurations for SMEFiT code
result_ID: LEP_linear_check

# absolute path where results are stored
result_path: /Users/eugeniacelada/Desktop/SMEFiT/

# path to common data
data_path:  /Users/eugeniacelada/Desktop/SMEFiT/smefit_database/commondata

# path to theory tables, default same as data path
theory_path:  /Users/eugeniacelada/Desktop/SMEFiT/smefit_database/theory

# pQCD order (LO or NLO)
order: NLO
use_theory_covmat: True
# SMEFT Expansion Order (NHO = Lambda^-2 , HO = Lambda^-4)
use_quad: False
# Set parameter bounds to previous SCAN result
bounds: Null
use_t0: False

#pairwise_fits: True
single_parameter_fits: True

# NS settings
nlive: 400
lepsilon: 0.05
target_evidence_unc: 0.5
target_post_unc: 0.5
frac_remain: 0.01
store_raw: false

efr: 0.005
ceff: True
toll: 0.5

# Datasets to include
datasets:

  # LEP
  # angular
  - LEP_eeWW_182GeV
  - LEP_eeWW_189GeV
  - LEP_eeWW_198GeV
  - LEP_eeWW_206GeV
  # EWPOs
  - LEP1_EWPOs_2006
  - LEP_alphaEW
  - LEP_Bhabha_2013
  - LEP_Brw_2013
  # VV
#  - ATLAS_WZ_13TeV_2016_mTWZ
#  - ATLAS_WW_13TeV_2016_memu
#  - CMS_WZ_13TeV_2016_pTZ

# Coefficients to fit
coefficients:

  ## TOP COEFFICIENTS ##
#   OpQM: {'min': -10.0, 'max': 10.0}
#   O3pQ3: {'min': -2.0, 'max': 2.0}
#   Opt: {'min': -25.0, 'max': 15.0}
#   OtW: {'min': -1.0, 'max': 1.0}
#   OtG: {'min': -1.0, 'max': 1.0}
#   Otp: {'min': -10.0, 'max': 5.0}
#   OtZ: {'min': -20.0, 'max': 10.0}

#   OQQ1: {'min': -200.0, 'max': 200.0}
#   OQQ8: {'min': -200.0, 'max': 200.0}
#   OQt1: {'min': -200.0, 'max': 200.0}
#   OQt8: {'min': -200.0, 'max': 200.0}
#   Ott1: {'min': -200.0, 'max': 200.0}

#   O81qq: {'min': -5, 'max': 5}
#   O11qq: {'min': -50, 'max': 50}
#   O83qq: {'min': -50, 'max': 50}
#   O13qq: {'min': -5, 'max': 5}
#   O8qt: {'min': -5, 'max': 5}
#   O1qt: {'min': -50, 'max': 50}
#   O8ut: {'min': -50, 'max': 50}
#   O1ut: {'min': -50, 'max': 50}
#   O8qu: {'min': -50, 'max': 50}
#   O1qu: {'min': -50, 'max': 50}
#   O8dt: {'min': -50, 'max': 50}
#   O1dt: {'min': -50, 'max': 50}
#   O8qd: {'min': -50, 'max': 50}
#   O1qd: {'min': -50, 'max': 50}

#   # # Do not enter with our flavor assumptions
#   # # Off: {'fixed': True,  'min': -35.0, 'max': 10.0}
#   # # ObW: {'fixed': True,  'min': -35.0, 'max': 10.0}
#   # # OQtQb1: {'fixed': True,  'min': -35.0, 'max': 10.0}
#   # # OQtQb8: {'fixed': True,  'min': -35.0, 'max': 10.0}
#   # # OQb1: {'fixed': True,  'min': -35.0, 'max': 10.0}
#   # # OQb8: {'fixed': True,  'min': -35.0, 'max': 10.0}
#   # # Otb1: {'fixed': True,  'min': -35.0, 'max': 10.0}
#   # # Otb8: {'fixed': True,  'min': -35.0, 'max': 10.0}

#   # ## HIGGS COEFFICIENTS ##
#   OpG: {'min': -1.0, 'max': 1.0}
#   OpB: {'min': -5.0, 'max': 5.0}
#   OpW: {'min': -10.0, 'max': 10}
   OpWB: {'min': -6, 'max': 6}
#   Opd: {'min': -15.0, 'max': 15.0}
   OpD: {'min': -13.0, 'max': 15.0}

   OpqMi: {'min': -3, 'max': 3}
   O3pq: {'min': -6, 'max': 6}
   Opui: {'min': -6.0, 'max': 6.0}
   Opdi: {'min': -6.0, 'max': 6.0}
#   Ocp: {'min': -0.44, 'max': 0.81}
#   Obp: {'min': -0.6, 'max': 0.22}
   Opl1: {'min': -3, 'max': 3.0}
   Opl2: {'constrain': [{'Opl1': 1}],'min': -3, 'max': 3.0}
   Opl3: {'constrain': [{'Opl1': 1}],'min': -3, 'max': 3.0}
   O3pl1: {'min': -3.0, 'max': 3.0}
   O3pl2: {'constrain': [{'O3pl1': 1}],'min': -3.0, 'max': 3.0}
   O3pl3: {'constrain': [{'O3pl1': 1}],'min': -3.0, 'max': 3.0}
   Ope: {'min': -7.0, 'max': 7.0}
   Opmu: {'constrain': [{'Ope': 1}],'min': -7.0, 'max': 7.0}
   Opta: {'constrain': [{'Ope': 1}],'min': -7.0, 'max': 7.0}
#   Otap: {'min': -0.28, 'max': 0.72}
   Oll: {'min': -1, 'max': 1}

#   ## DIBOSON COEFFICIENTS ##
   OWWW: {'min': -2.0, 'max': 2.0}
#   OW: {'constrain': [{'OpD': -5.1754},{'OpWB': -11.2835}], 'min': -0.80, 'max': 0.25}
#   OB: {'constrain': {'OpD': 2.8209}, 'value':, 'min': -0.85, 'max': 0.30}

rot_to_fit_basis: null

[giacomomagni]

I'm able o run your runcard (modulo trivial chanhges in paths).

Can you try to reinstall?

[ecelada]

It's running now after reinstallation... thanks

[giacomomagni]

No problem, sorry for the delay in the answer!

[ecelada]

I actually got a new error after a couple of minutes of running...

Traceback (most recent call last): File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/pandas/core/indexes/base.py", line 3803, in get_loc return self._engine.get_loc(casted_key) File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc File "pandas/_libs/index.pyx", line 165, in pandas._libs.index.IndexEngine.get_loc File "pandas/_libs/hashtable_class_helper.pxi", line 5745, in pandas._libs.hashtable.PyObjectHashTable.get_item File "pandas/_libs/hashtable_class_helper.pxi", line 5753, in pandas._libs.hashtable.PyObjectHashTable.get_item KeyError: 'Opl1'

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/Users/eugeniacelada/Library/Python/3.10/bin/smefit", line 8, in sys.exit(base_command()) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/click/core.py", line 1130, in call return self.main(args, kwargs) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, ctx.params) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/click/core.py", line 760, in invoke return __callback(args, *kwargs) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/smefit/cli/init.py", line 81, in nested_sampling runner.run_analysis("NS") File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/smefit/runner.py", line 215, in run_analysis self.single_parameter_analysis(optimizer) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/smefit/runner.py", line 179, in single_parameter_analysis self.ultranest(single_coeff_config) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/smefit/runner.py", line 131, in ultranest opt.run_sampling() File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/smefit/optimize/ultranest.py", line 243, in run_sampling sampler = ultranest.ReactiveNestedSampler( File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/ultranest/integrator.py", line 1196, in init if not self._check_likelihood_function(transform, loglike, num_test_samples): File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/ultranest/integrator.py", line 1258, in _check_likelihood_function logl = loglike(p) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/ultranest/utils.py", line 134, in vectorized return np.asarray([function(arg) for arg in args]) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/ultranest/utils.py", line 134, in return np.asarray([function(arg) for arg in args]) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/smefit/optimize/ultranest.py", line 215, in gaussian_loglikelihood return -0.5 self.chi2_func_ns(hypercube) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/smefit/optimize/ultranest.py", line 196, in chi2_func_ns self.coefficients.set_constraints() File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/smefit/coefficients.py", line 236, in set_constraints free_dofs = [ File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/smefit/coefficients.py", line 237, in self._table.at[fixed_name, "value"] File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/pandas/core/indexing.py", line 2431, in getitem return super().getitem(key) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/pandas/core/indexing.py", line 2382, in getitem return self.obj._get_value(*key, takeable=self._takeable) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/pandas/core/frame.py", line 3922, in _get_value row = self.index.get_loc(index) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/pandas/core/indexes/base.py", line 3805, in get_loc raise KeyError(key) from err KeyError: 'Opl1'

Did you run my card til the end?

[giacomomagni]

I don't have all your datasets, sorry, which branch of the smefit_database shall I use?

[ecelada]

Right, they are form smefit_ewpo

[giacomomagni]

do you have a link?

[ecelada]

https://github.com/LHCfitNikhef/smefit_ewpo But it's private I'm not sure you have access, do you?

[ecelada]

However the problem doesn't seem related to the dataset, because they run seemingly fine with single_parameter_fit: False

[giacomomagni]

I found them. Okay the problem it that you have constrained coefficients and right now the code when you use: single_parameter_fits: True fits all the coefficients switching them on one by one. Ofc here this is not what you want, sicne you want to loop on the coeffiecnts and also keep the contrains.

[giacomomagni]

In your case there is an easy fix, because you have really simple constrain, in general the situation will be more complicated.. Because in general if have multiple coefficients entering in a constrain you can't propagate this constrain in a single parameter fit. So if you want to get some results now, I can provide an easy fix.

[ecelada]

Alright, thank you! what's the easy fix?

[giacomomagni]

So add these lines:

            # skip contrained coeffs
            if 'constrain' in config['coefficients'][coeff]:
                print(f"Skipping contrained coefficient {coeff}")
                continue

            # propagate simple constrains
            new_coeff_config = {}
            for coeff2, vals in config['coefficients'].items():
                if 'constrain' in vals:
                    constrain =  vals['constrain']

                    # only single coefficient constrain are supported
                    if len(constrain) > 1:
                        raise ValueError("Contrains are tooo long")

                    if coeff in constrain[0]:
                        new_coeff_config[coeff2] = vals            
            # add regular coeff
            new_coeff_config[coeff] = config["coefficients"][coeff]
            single_coeff_config["coefficients"] = new_coeff_config

instead of:

https://github.com/LHCfitNikhef/smefit_release/blob/d37742cd3d0ea8a77dd713ffeb8532267c904ff4/src/smefit/runner.py#L191

but keep in mind that is not a tested and quite rough code... You should double check. I'll try to implement a proper fix if possible, and let you know. So far we were not supporting constrain in single parameter fits...

I will not available next week, so will reply only later sorry.

[ecelada]

Ok thanks a lot!

[ecelada]

Unfortunately the problem persists :(

Traceback (most recent call last): File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/pandas/core/indexes/base.py", line 3803, in get_loc return self._engine.get_loc(casted_key) File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc File "pandas/_libs/index.pyx", line 165, in pandas._libs.index.IndexEngine.get_loc File "pandas/_libs/hashtable_class_helper.pxi", line 5745, in pandas._libs.hashtable.PyObjectHashTable.get_item File "pandas/_libs/hashtable_class_helper.pxi", line 5753, in pandas._libs.hashtable.PyObjectHashTable.get_item KeyError: 'Opl1'

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/Users/eugeniacelada/Library/Python/3.10/bin/smefit", line 8, in sys.exit(base_command()) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/click/core.py", line 1130, in call return self.main(args, kwargs) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, ctx.params) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/click/core.py", line 760, in invoke return __callback(args, *kwargs) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/smefit/cli/init.py", line 81, in nested_sampling runner.run_analysis("NS") File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/smefit/runner.py", line 215, in run_analysis self.single_parameter_analysis(optimizer) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/smefit/runner.py", line 179, in single_parameter_analysis self.ultranest(single_coeff_config) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/smefit/runner.py", line 131, in ultranest opt.run_sampling() File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/smefit/optimize/ultranest.py", line 243, in run_sampling sampler = ultranest.ReactiveNestedSampler( File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/ultranest/integrator.py", line 1196, in init if not self._check_likelihood_function(transform, loglike, num_test_samples): File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/ultranest/integrator.py", line 1258, in _check_likelihood_function logl = loglike(p) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/ultranest/utils.py", line 134, in vectorized return np.asarray([function(arg) for arg in args]) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/ultranest/utils.py", line 134, in return np.asarray([function(arg) for arg in args]) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/smefit/optimize/ultranest.py", line 215, in gaussian_loglikelihood return -0.5 self.chi2_func_ns(hypercube) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/smefit/optimize/ultranest.py", line 196, in chi2_func_ns self.coefficients.set_constraints() File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/smefit/coefficients.py", line 236, in set_constraints free_dofs = [ File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/smefit/coefficients.py", line 237, in self._table.at[fixed_name, "value"] File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/pandas/core/indexing.py", line 2431, in getitem return super().getitem(key) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/pandas/core/indexing.py", line 2382, in getitem return self.obj._get_value(*key, takeable=self._takeable) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/pandas/core/frame.py", line 3922, in _get_value row = self.index.get_loc(index) File "/Users/eugeniacelada/Library/Python/3.10/lib/python/site-packages/pandas/core/indexes/base.py", line 3805, in get_loc raise KeyError(key) from err KeyError: 'Opl1'

[giacomomagni]

Let's text on slack