Improvement on flattening function

[Ameyanagi]

This is a proposed improvement for the flattening function. Currently, the pre-edge flattening function uses a quadric function, regardless of the order selected for pre-edge orders. This occasionally results in artifacts that alter the vertical shift in the flattened spectra, which is undesirable for visual comparisons.

I suggest utilizing the computed pre-edge and post-edge lines to determine the flattened spectra. This algorithm is already implemented in xraytsubaki.

Previous approach:

Proposed approach:

[newville]

@Ameyanagi Thanks -- I think that would be a fine change. It might be useful to see a few more examples. I think the main usage for flattening is fluorescence data where there can be a noticeable drift (or even "swoop") in If/I0 well above the edge that is not removed by normalization alone.

Or: maybe allow for both approaches?

[Ameyanagi]

@newville

Sorry for taking time, I plotted some example foils with "Athena", "Larch", and "Proposed Method".

I read the files with read_athena and did the following before plotting.

e0 = group.athena_params.bkg.e0
pre1 = group.athena_params.bkg.pre1
pre2 = group.athena_params.bkg.pre2
nor1 = group.athena_params.bkg.nor1
nor2 = group.athena_params.bkg.nor2
nnorm = group.athena_params.bkg.nnorm
step = group.athena_params.bkg.step

pre_edge(group, e0=e0, step=step, pre1=pre1, pre2=pre2, norm1=nor1, norm2=nor2, nnorm=nnorm, make_flat=True)

The results are the following. It is not an exact match, but the proposed flattening function seems be more closer to 1. All of the method behave horrible after norm2.

I didn't have much time to optimize all the parameters for each foil.

[Ameyanagi]

@newville

I found the source in Demeter. I propose to use the algorithm I made a pull request, to match the concept with Athena.

Ifeffit backend: https://github.com/bruceravel/demeter/blob/edfaa0904fa30426cf303b5933450edf22a426ec/lib/Demeter/templates/process/ifeffit/flatten_set.tmpl

Larch backend: https://github.com/bruceravel/demeter/blob/edfaa0904fa30426cf303b5933450edf22a426ec/lib/Demeter/templates/process/larch/flatten_set.tmpl

The concept of Athena was the same as the algorithm I proposed, but the calculation is slightly different, and it has a bug on Athena when calculating with a normalization order of 3. (At least the implementation is different from the comments.) I leverage the pre_edge and post_edge stored in the Group. I think this is more general because it can handle the Victoreen functions and high-order polynomial that might be used for post_edge.

The Athena code only implements the flattening function up to the second order of the polynomial, and I think this is not what the code wants to do, based on the comments.

One thing that we can say is that the current implementation of Larch is different from Athena.

I think we should use the same concept as Athena but use my algorithm so that we don't make a new bug in the future when we increase the order of polynomials in the pre_edge or post_edge.

[newville]

Ok, thanks. I think it I'd okay to be a little different from Athena, well, if it is better ;)

[newville]

@Ameyanagi I'll merge this, but may make some mods...