In-situ linear fit from plot context menu behaves oddly

[smk78]

Testing 5.0.1RC on x64 under W10.

Right-clicking on a plot and selecting a dataset gives the option for a linear fit. The fit parameters box that opens behaves oddly.

The 'range (linear scale)' boxes aren't editable and seem to default to 0000001 and 9999984. Underneath the 'Fit range of log10(x)' boxes are editable but do not default to the Q-range of the data. Given that the data is always displayed with unlogged axis labels it is not clear what the purpose of these two sets of boxes is. Surely the fitting range is always x?

Clicking 'Fit' does however generate a linear fit and populate the fit parameter boxes. But I'm not sure I trust the values. Fitting a random normally-tailing off dataset that had an I(Q=0.007)=642 returned a=-2.52 (which was roughly the gradient) but b=-3.01! How can the intercept be so small and negative? If it was +3.01 I might have assumed it was log10(b), but not if it is -3.01.

[rozyczko]

I agree with you. However, the behaviour is identical in 4.x so probably should be changed in both. In 4.x the linear range text boxes are also not editable and the intercept is negative. The example below is the sphere/sphere model with default parameters with linear fit applied.

[butlerpd]

There is quite a bit of confusion here I think. This is what I consider critical functionality but which we have never had resources to do well so it is a kludge done a long time ago using mostly existing stuff with minimal extra work. Hence it is clunky and ugly. However it is mostly correct I think in 4.x.

The point of this is that there are a number of useful parameters that can be extracted from a linear fit to transformed data (transformed so that the region of interest becomes a straight line). The prime example of course is the famous Guinier plot but also things like the Porod plot etc. If you choose the "appropriate" region of the transformed plot you can fit a straight line and extract useful info (the Rg for example). Using this with the log(10) log(10) scale is not transformed data and pointless as far as I can tell and likely to lead to some very silly numbers. For this to be useful you need to transform the data by using the "change scale" option first (and choose something useful like Guinier) then apply the linear fit to the appropriate region.

Thus to answer the question about why two sets of ranges, for a Guinier the range will be in q^2 units but the user may want to know what q that covers. The point of both sets of boxes was to allow the user to specify the range in whatever "units" they wished -- those boxes must however be tied!!! changing the q^2 min also must changes q min, and vice versa.

So as far as I can tell, the main problem with this feature in 5.x is that the x ranges are not tied properly? there may be some other issues but I'll have to test more to be sure.

[butlerpd]

After fairly thorough testing I think the implementation was hampered by some confusion as to the purpose as mentioned before. The testing was done with the ESS_GUI build of Dec 23 and compared to the 4.2.2 release version. There are really 3 issues that should be addressed. The main critical issue is:

• The SAS transformations should show the appropriate data for the linear fit .. otherwise the whole exercise is pointless! 4.x did have them. So for Guinier it would be Rg and I0, for xsection it would be Rgc for porod the background etc .. actually the Porod should also give the Porod constant which I think is just the intercept (4.x should have this as well) If the transform is not recognized then the user will need to convert the slope and intercept to extract the value if any. This was explained in the 4.x help documentation but removed from the 5.x documentation.

The other two rather major issues are

• As mentioned in the original issue, the non-editable boxes look editable and make for a very confusing experience. Non-editable boxes are labels and should be obviously not editable (at the very least the same background color as the panel, usually grey). A lot of work over the years went into doing that in 4.x due to user confusion.
• When rescaling a transformed plot, the original range of the y axis is for the non transformed axis rather than the transformed one. If the user just clicks ok the graph will go completely out of range. To see this take a data set. Then use the scale menu to transform to Guinier. Now use the set Range menu item to rescale the axes. The initial values in the y range will be the min and max I not the min max of ln(I) (quite a huge difference). Likewise the initial values in the x range are the min and max of q not q^2 (less dramatic of a difference). This works properly in 4.x.

There is a unique confusion with log10 transformations since it can be “transformed” graphically while keeping the non-transformed values on the scale (i.e. 10^3 instead of 3 etc.) which is extremely useful for looking at scattering data in general but a bit confusing for the crude linear fit module of SasView. That makes the intercept at log10(I)=0 (I=1) visually not so obvious but I am not aware of any utility to that intercept. The slope however provides a quick and dirty way of estimating the slope of the data - often at low q the question might be: is that a q^-2? Or q^-4? Etc… This has always been so and while it would be nice to figure out how to make more obvious, I don’t think anything needs to be done for an imminent release.

[rozyczko]

Added missing analysis for Guinier/Porod (code and docs). Made the text boxes more clear wrt. editability.

Currently in ESS_GUI_5.0.1_fixes

[smk78]

Tested in 5.0.1 Build 321 and locally-built from ESS_GUI branch. Working a lot better now. But:

• in light of the discussion above, can we add to the text at the top of the Linear Fit popup window so it says, say, Perform fit for y(x)=ax+b on transformed data (use Change Scale first)?
• why is it that the Fit range of x min value is often negative? If I transform to Guinier (where x=Q^2) or Porod (where x=Q^4), how can this be? I was using the AOT_Microemulsion-Drop_Contrast.xml & ISIS_Polymer_Blend_RT2.xml test datasets.
• And lastly, there are no derived parameters for the Porod and Kratky transformations (like there are for the Guinier and X-Sect Guinier).

[butlerpd]

To answer point 2: x always goes from -inf to + inf. For Guinier and Porod there should of course be no data in the range -inf to 0 :-) I think the default for original x is the data min max while the default x (transformed) data is the plot min max ..... which will go negative if the first point is near enough to zero.

As to the last point: 4.x give the Background (which is just the slope) but it would be nice to also give the Porod Constant which I think is the intercept if I remember right? (This last is NOT currently done in 4.x as far as I can tell). As for Kratky I fear my memory fails me as to how that got added but suspect some people (maybe bio people? but then would not be linear) wanted to be able to see a quick Kratky. Interesting question: Are there useful parameters one can obtain from that? The Kuhn length maybe? I must admit I never use Kratky plots so not sure .. however I don't think 4.x does anything with a fit there either.

[smk78]

Actually, on reflection:

• Porod: Q^4 I(Q) vs Q^n. The intercept is a constant = Kp.Sv. But unless you know one or other of the two terms on the RHS you cant derive the other. So just providing a and b is ok.
• Kratky: Q^2 I(Q) vs Q. Similarly, in the high-Q limit, the intercept is a constant = (2NV^2delta-rho^2)/Rg^2. But again you need more information to make use of a & b.

The one we could do is a Zimm transformation: I(Q)^-1 vs Q^2 where not only does b have meaning, but a = (Rg^2/3).b

[butlerpd]

exactly but isn't Kp.Sv the porod constant that you need for example in the invariant calculation? Labeling it as such could be useful for the more novice user? ... or the not so novice since we had to think about it a while :-) But agree we should not worry about it for now.

[smk78]

Just need to be careful about our definitions of the Porod Constant (as in, whether or not one or the other includes Sv).