search

DARPA-ASKEM / sciml-service

Simulation Service provides an interface and job runner for ASKEM models.
MIT License
3 stars 1 forks source link

Fail gracefully when `calibrate` does not converge #36

Closed fivegrant closed 1 year ago

fivegrant commented 1 year ago

A model can end up taking endless amounts of time trying to fit a data. #34 will help remedy this a little bit by allowing the user to peek inside the job, but there needs to be some kind of failure instead of no return. A few tweaks that could help are

calibrate is brittle so the failures need to be normalized so they can be handled better by the HMI.

mj3cheun commented 1 year ago

in general, it would be great if specific actionable (tying back to the input data) errors are added to the calibration output. currently when a calibration fails, it will just return a list of parameters with null values and it is up to the user to guess what caused the problem and fix it.

specifically there are certain types of errors i suspect might contribute to failure to converge, they are:

  1. bad initial parameters settings
  2. not enough feature mappings
  3. dataset / model mismatch

would be great if its possible to get error messages that are helpful in narrowing down the specific cause

fivegrant commented 1 year ago

Not sure how relevant this issue is after the refactor. Closing for now