UCL / TLOmodel

Epidemiology modelling framework for the Thanzi la Onse project
https://www.tlomodel.org/
MIT License
11 stars 5 forks source link

healthseekingbehaviour linear model causing errors #786

Closed tdm32 closed 1 year ago

tdm32 commented 1 year ago

Error appearing in long runs (200k) after approx 2 hours:


    hsb_model.predict(
  File "/TLOmodel/src/tlo/lm.py", line 417, in predict
    column_resolvers = self._get_column_resolvers(df, **kwargs)
  File "/TLOmodel/src/tlo/lm.py", line 374, in _get_column_resolvers
    column_resolvers[cleaned_name] = col.astype(dtype)```

``` File "/usr/local/lib/python3.8/site-packages/pandas/core/arrays/categorical.py", line 422, in astype
    raise ValueError("Cannot convert float NaN to integer")
ValueError: Cannot convert float NaN to integer```
tbhallett commented 1 year ago

Looks like something in simulation.populations.props has gone to NaN.

Any ideas what it would be....?

Perhaps a quick-fix would be to put some safety in around the LinearModel evaluation so that it fails more gracefully or more informatively under such circumstances.

tbhallett commented 1 year ago

@tdm32 can you identify the run used (script, branch, commit)

tdm32 commented 1 year ago

@tdm32 can you identify the run used (script, branch, commit)

It's on one of my azure runs. I can get the seed and parameter sets used and run locally to try and reproduce the error here - perhaps with some additional outputs from the lm to help me debug. So far it has only happened in one run from each set

tdm32 commented 1 year ago

@tdm32 can you identify the run used (script, branch, commit)

Job: scenario_3-2022-12-02T183108Z draw_1-run_9 branch tara_tb_transmission

matt-graham commented 1 year ago

It looks like this might the same problem that was fixed by #715 which was merged in to master after tara_tb_transmission was branched off. Either merging current master in to the branch or cherry picking the relevant commits from the PR branch might therefore fix.

tdm32 commented 1 year ago

It looks like this might the same problem that was fixed by #715 which was merged in to master after tara_tb_transmission was branched off. Either merging current master in to the branch or cherry picking the relevant commits from the PR branch might therefore fix.

Thanks Matt - shall I close this issue now?