Allow single step option during fitting (Trac #31)

[butlerpd]

Richard Heenan points out that it can be very useful when fits go awry to be able to single step through each iteration, with a rewind option.

Migrated from http://trac.sasview.org/ticket/31

{
    "status": "new",
    "changetime": "2017-02-28T23:29:38",
    "_ts": "2017-02-28 23:29:38.775807+00:00",
    "description": "Richard Heenan points out that it can be very useful when fits go awry to be able to single step through each iteration, with a rewind option.",
    "reporter": "butlerpd",
    "cc": "",
    "resolution": "",
    "workpackage": "SasView Fitting Redesign",
    "time": "2012-03-29T22:19:04",
    "component": "SasView",
    "summary": "Allow single step option during fitting",
    "priority": "major",
    "keywords": "",
    "milestone": "SasView WishList",
    "owner": "",
    "type": "enhancement"
}

[butlerpd]

Trac update at 2012/05/06 16:07:28: butlerpd changed milestone from "" to "WishList"

[butlerpd]

Trac update at 2015/01/30 00:41:32: butler changed workpackage from "" to "SasView Fitting Redesign"

[ajj]

Trac update at 2017/01/17 13:05:47: ajj changed component from "SansView" to "SasView"

[krzywon]

This ticket came up today during our training class at NIST. The biggest hurdle the students are facing is no longer writing the model structure, it is debugging the mathematics, especially in more complex systems. The suggestion in this ticket may not be the best way, but some form of debugging tool is needed when writing plugin models.

[butlerpd]

perhaps we could implement a logger function for the C code that can pass back to the console whatever it is passed? maybe not elegant.... but?

[pkienzle]

I mostly use sascomp from the command line for debugging models.

It turns out you can run it from >tool >python shell/editor as well:

import sasmodels.compare
def sascomp(s): sasmodels.compare.main(*s.split())
sascomp("sphere radius=500,600") # compares two radii of sphere on the same plot
sascomp("sphere,cylinder length=4/3*radius -highq") # compare sphere to same volume cylinder
cd ~/.sasview/plugin_model
sascomp("newmodel.py")  # runs a model in the plugin directory
sascomp("-h") # Show the many options available to sascomp

Unfortunately you can't use --edit for the interactive model viewer that lets you drag parameters with a slider since sasview no longer provides wx.

The other thing I use is sasmodels/explore/realspace.py. Using points uniformly selected from the shape I can compute a monte carlo approximation to the model and compare it against the analytic form implemented in sasview. This option isn't available inside the sasview gui.

[smk78]

Also see https://github.com/SasView/sasview/issues/1800

[butlerpd]

To clarify a bit there are (so far) 2 critical issues identified here (none of which are about validation of the model - i.e. is it giving the correct answer compared to some known "standard").

1. "compile" errors (or code errors). Users have a lot of problems typing everything exactly correctly with all the parenthesis paired properly and the commands spelled correctly etc. This would be what #1800 is speaking to for pure python models. It seems that C compile errors are providing better feedback in the console but I have not checked that. It certainly needs to work for both the C code and the python code.
2. A complicated model runs perfectly with no errors BUT is giving the wrong answer. This is a very common occurrence. How to find where that error is coming from? For Python models it is a simple matter of putting print statements in as they will be displayed on the console. However printf does not seem to print to the console and there is no simple way to access all the C variables in python for printing out.... and complicated models are likely to be in C. This is a huge issue it turns out. If there were an easy way to allow printf to print to the console log that would fix this problem.

There may be other issues I'm forgetting but these are the ones I noticed both in preparing the demo and working with the model writing group.

[pkienzle]

You can put printf statements into the C code so long as you don't run it on the gpu. Easy enough to do with sascomp from the terminal but this requires a python development environment.

You could set up the sasview application to run python scripts. This would be useful for people who want to do batch processing on files but don't have a python development environment set up. Add the -m option similar to the python interpreter and you could test your model using sasview -m sasmodels.compare model.py. The printf output should then show up in the terminal.

You could probably run the model within the sasview gui and capture the output to the log window but it'll take a bit of fiddling. Maybe it is as simple as redirecting sys.stdout to a string before running the model. Or maybe you will have to run it as a separate process and capture the process output.

[RichardHeenan]

The ability to launch a "debug my model" process or window might be good ask user for a single Q value, or list of Q values, then print out everything that comes back from compilation and run output - the main results, plus perhaps, optionally, the whole long list that the model returns. This would be a half way house to what "show plot" might do. May be very helpful when we try to make more complex sum & product combination models, where sasview needs some work still. My original request on this ticket was to just run one iteration of a "fit" rather than run to convergence, as that might also help in a debug situation. (Personally, because of all the difficulties, I always make new models in the main code of my developer build not the plugin directory, print( ) or printf() in the models appears in the console window, so maybe I'm not the best to comment on this as I never use the gui model editor.)