search

CSULA-Qdots / filter

Data filtering for the CSULA quantum-dot spectroscopy experiment
3 stars 2 forks source link

Consider other filters than moving average #12

Open jbouzanquet opened 11 years ago

jbouzanquet commented 11 years ago

I was just reading an article on climate change, and one of the graphs in the article intrigued me. They show a curve rather like we want to produce for our "filtered" or "smoothed" data, but use what they call a binomial filter. Google produces a page with a simple description of this filter. The binomial filter appears to be a moving average weighted by binomial coefficients. Is it worth trying it instead of the moving average?

blackdjinn commented 11 years ago

Yes. We should consider them. These are doing something different in terms of how they transform the data, however. These behave by taking a noisy signal and returning a less noisy signal, so the original data points are replaced. The current filter is designed to 'cherry pick' the data by selecting points that are less noise-like. No real problem with either method, tho.

jbouzanquet commented 11 years ago

Something that also came to mind is if we use a binomial filter with the weights reflected across the axes 1/4 way from the center. Basically the far edges count more. I'm not sure I'm thinking through that right, particularly with my headache right now. I'm trying to come up with a way to eliminate the spikes. See the spreadsheets I made from generated data that still have some of the spikes remaining.

blackdjinn commented 11 years ago

No... the binomial puts more emphasis on the local points.

Think about it this way:

  1. Run a Fourier transform on the data.
  2. Throw away all but the lowest frequency components.
  3. Plot the resulting function.

The results should be a LOT closer to the actual signal than the noisy ones we have. The whole running-average or binomial methods are ways of stripping the high-frequency noise components out of the data signal.