Background removal and normalization

[matthewcarbone]

Zhu's summary:

Background removing and normalization: subtract the background by fitting the pre-edge region to a line, so the pre-edge region is pretty much zeroed. And then the intensity through whole range is divided by the standard deviation so they all have similar variation (useful for some algorithm).

@zhul9311 question: there is a standard way of doing this, do you know what it is/is there a paper we can reference that describes the implementation?

[zleung9]

I didn't know of a specific paper that uses this method. I learned it from Fundamentals of XAFS by Matthew Newville (link below). https://millenia.cars.aps.anl.gov/xraylarch/downloads/2018Workshop/NewvilleEXAFS_RIMG78_ColorPreprint.pdf Below is a quote from Chapter 5.1.

That is a linear fit to the pre-edge range of the measured spectrum is found, and subtracted. In some cases, a so-called Victoreen pre-edge function (in which one fits a line to μ(E)*E^n for some value of n, typically 1, 2 or 3).

This method is implemented in larch package, in which a line is fit to the mu(E)*E^t the pre-edge region (by default t=0). https://xraypy.github.io/xraylarch/xafs_preedge.html

Although I only implemented linear fit, it might be a good idea to implement Victoreen method and set the default to linear.

[matthewcarbone]

@zhul9311 that's perfect, thank you. What you're proposing sounds good to me. If you'd like to take a shot at it you're welcome to, just assign yourself to the issue so everyone knows and nobody steps on your toes by accident.

[zleung9]

Sure. I couldn't find the assign yourself button as shown in this help page: https://docs.github.com/en/issues/tracking-your-work-with-issues/assigning-issues-and-pull-requests-to-other-github-users. Could you assign it to me?

There is a newbie question: if I want to add this implementation to the repo, what is the standard way of doing this? Can I just create a new python file to my local repo and create a pull request?

(BTW I will reply other issues later, they are in queue)

[matthewcarbone]

Sure no problem, just assigned it to you.

There is a newbie question: if I want to add this implementation to the repo, what is the standard way of doing this? Can I just create a new python file to my local repo and create a pull request?

No worries. Yes that's correct, but I wouldn't create a new python file, I think we can add these simple operations to the operations.py file I already created. I think you've already done this before via PR. It's basically no different than what you've done before. Happy to have a chat about it if there's anything that's still confusing.

[zleung9]

OK. Thank you, that is very clear. I will add it and discuss through pull request.

[zleung9]

17 Added the background reduction operator using your template.