Detailed location reports for "Computing model run failure" errors

[fentonscode]

Description When a model fails with "Computing model run failure" there is no indication where the failure came from. With multiple custom plugins - such as extractors, mixers, movers and iterators all potentially used in parallel this means a long search or adding redundant spam calls to raise or print to add tracking when run from the command line.

A good upgrade would be either using any available Python exception / stack trace information or if possible just the module / file from which the failure occurred.

[chryswoods]

Yes, you're right that the exception information is lost when running multiple models in parallel, which is a pain. You do get this information as a stack trace with module/file when running a single model, so as a start, you can re-run the failing designs as a single run to debug.

I'll see if I can capture the error and add it to the end of the "output.txt.bz2" file for the model runs that fail. I don't want to print too much to the main screen as it will spam things.

[chryswoods]

I've added this feature. The exception will be written to the corresponding output.txt.bz2 file for the model run. I've added this as part of the feature_named_stages branch, which is quite unstable (there is a lot of work needed with how I map the counts of different stages now that each demographic can have completely different pathways with stages that can have completely different names).

If you are interested, the specific change for this feature is just the diff from devel for _worker.py and _console.py

https://github.com/metawards/MetaWards/commit/b340ea5c8cea1dc8b428fed8c8f824556aa34eda

[chryswoods]

This will be merged into devel with the attached pull request. If accepted, this will close this issue.