`petab.calculate_llh` and `petab.calculate_chi2` with observableParameters

[paulflang]

I'd like to report that for me petab.calculate_llh and petab.calculate_chi2 fail with a KeyError when the measurement table contains an observableParameters column. When removing the observableParameters column it works.

[dweindl]

Hi @paulflang , thanks for reporting. Confirming as bug. Fixed in #451 .

[paulflang]

Thanks @dweindl !

[paulflang]

A similar problem occurs with noiseParameters, except that this cannot be fixed by removing the noiseParameters column:

TypeError                                 Traceback (most recent call last)
/media/sf_DPhil_Project/Project07_Parameter Fitting/PEtab/petab/calculate.py in evaluate_noise_formula(measurement, noise_formulas, parameter_df, simulation)
    186     try:
--> 187         noise_value = float(noise_value)
    188     except TypeError:

~/venvs/std/lib/python3.8/site-packages/sympy/core/expr.py in __float__(self)
    324             raise TypeError("can't convert complex to float")
--> 325         raise TypeError("can't convert expression to float")
    326 

TypeError: can't convert expression to float

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
<ipython-input-25-3d803dd96edb> in <module>
     26 problem = core.DisFitProblem(PETAB_YAML)
     27 problem.write_jl_file()
---> 28 problem.optimize()
     29 problem.plot_results('c0', path='plot.pdf')
     30 problem.write_results()

/media/sf_DPhil_Project/Project07_Parameter Fitting/df_software/DisFit/DisFit/core.py in optimize(self)
    194         # Todo: remove the removal of the `observableParamters` column once the bug it causes in petab.calculate_llh is fixed.
    195         cols = [not b for b in self.petab_problem.measurement_df.columns.isin(['observableParameters', 'noiseParameters'])]
--> 196         self._results['fval'] = -petab.calculate_llh(self.petab_problem.measurement_df.loc[:, cols],
    197             self.petab_problem.simulation_df.rename(columns={'measurement': 'simulation'}),
    198             self.petab_problem.observable_df, self.petab_problem.parameter_df) # self._results_all['objective_value'][self._best_iter]

/media/sf_DPhil_Project/Project07_Parameter Fitting/PEtab/petab/calculate.py in calculate_llh(measurement_dfs, simulation_dfs, observable_dfs, parameter_dfs)
    271     for (measurement_df, simulation_df, observable_df, parameter_df) in zip(
    272             measurement_dfs, simulation_dfs, observable_dfs, parameter_dfs):
--> 273         _llh = calculate_llh_for_table(
    274             measurement_df, simulation_df, observable_df, parameter_df)
    275         llhs.append(_llh)

/media/sf_DPhil_Project/Project07_Parameter Fitting/PEtab/petab/calculate.py in calculate_llh_for_table(measurement_df, simulation_df, observable_df, parameter_df)
    312 
    313         # get noise standard deviation
--> 314         noise_value = evaluate_noise_formula(
    315             row, noise_formulas, parameter_df, petab.scale(simulation, scale))
    316 

/media/sf_DPhil_Project/Project07_Parameter Fitting/PEtab/petab/calculate.py in evaluate_noise_formula(measurement, noise_formulas, parameter_df, simulation)
    187         noise_value = float(noise_value)
    188     except TypeError:
--> 189         raise TypeError(
    190             f"Cannot replace all parameters in noise formula {noise_value} "
    191             f"for observable {observable_id}.")

TypeError: Cannot replace all parameters in noise formula noiseParameter1_obs_a + noiseParameter2_obs_a for observable obs_a.

[dweindl]

TypeError: Cannot replace all parameters in noise formula noiseParameter1_obs_a + noiseParameter2_obs_a for observable obs_a.

@paulflang Do you have those parameters defined in the noiseParameters column of you respective measurements?

[paulflang]

This happens for instance with the petab_test_suite case 0014, if I remember correctly. The noiseParamerers column is in the tsv file, but the error code shows what (unsurprisingly) happens when removing it from the measurement_df that us fed into petab.calculate_llh.

[dweindl]

Do I get it correctly: You called petab.calculate_llh with a simulation table that did not have a noiseParameters column (or an empty one)? This should fail, as without noise parameters it is not possible to compute the likelihood.

[paulflang]

Oh, I see. Looks like both, the measurement and the simulation table need to contain the noiseParameters then.