search

jamoma / jamoma2

A header-only C++ library for building dynamic and reflexive systems with an emphasis on audio and media.
MIT License
30 stars 6 forks source link

WhiteNoise class needs Unit Test #32

Open tap opened 9 years ago

tap commented 9 years ago

Not sure how to approach this one. Any ideas @lossius ?

lossius commented 9 years ago

The same algorithm, with the same initial seed value, should produce the same series of numbers in octave/matlab. That will help testing for consistency. As for whether the algorithm actually produce white noise, the best would be to just monitor and hear/analyse the resulting sound over time once, and see that it does not have biases. Probably the algorithm will be based on a snippet of code from some trustworthy source, so once we have played and listened to it and we are sure that we did not do crazy mistakes in the port and it sounds OK, we could trust that the algorithm has the properties that it is claimed to have in the source literature.

tap commented 9 years ago

I guess I was thinking that rather than test for specific numbers a test for this would be something along the following lines:

  1. Take an FFT (probably do a bunch of runs and then average a bunch of FFTs)
  2. Determine the mean and std deviation
  3. Assert that there are no outliers

But perhaps this is faulty in some way?

lossius commented 9 years ago

That is unreliable. You could be unfortunate to pick a seed that temporarily leave the mean and standard-deviation to be slightly different. It's in the nature of random processes not only to have a mean value, but also to deviate.

It's better to do this once manually on a longer sample to verify that the algorithm produce sane results, but rather use a fixed seed in the unit test.

nwolek commented 8 years ago

Found these today: http://stats.stackexchange.com/questions/7074/formal-statistical-test-for-whether-a-process-is-a-white-noise https://en.wikipedia.org/wiki/Ljung%E2%80%93Box_test

Apparently this sort of test has importance in financial data.

lossius commented 8 years ago

Yes, but I don't think those tests have much relevance to us. In statistics and numerical computation science new methods (functions) might be developed that are assumed to produce white noise. These new functions need to be verified for whether they actually do produce white noise.

However, in an implementation, when a well-known, researched and documented numerical method is used, the question is no longer whether it produces white noise or not, but rather whether it is implemented correctly or not. For that a seed approach is fine.