search

WireCell / wire-cell-gen

Wire Cell packages that can generate things like wires, cells, depositions, diffusions, slices, frames, ...
0 stars 1 forks source link

Speed up noise generation. #6

Closed brettviren closed 6 years ago

brettviren commented 6 years ago

Right now the noise is generated in a correct but slow way (although faster than yet more naive methods).

1) A real-valued noise amplitude spectrum of N=100 bins is provided.

2) The N=100 spectrum is interpolated to N=5000 (just for the example of 2MHz sampling over 5ms of time and specified only up to the 1MHz Nyquist freq)

3) 5000 randoms are drawn to Rayleigh-sample the amplitude and another 5000 randoms to uniform-sample the complex phase of each frequency bin.

4) An inverse FFT is applied to the result to produce 10k sample time series for that channel.

There can not be any new information about the amplitude magnitude added in this procedure and any choice of random phase leaves the autocorrelation function invariant.

So, it seems that reversing the order of interpolation and random-sampling steps would provide the same final result. This would require 50x fewer randoms at the cost of a 2x larger interpolation (100 real vs 100 complex values). And this allows the interpolation to be done by padding zero-amplitude high frequency bins to the sampled spectrum in order to pad from 100 to 5000 and just before it is consumed by the inverse-FFT.

Reversing these steps will have implications on the API/contract between NoiseSource and the noise model classes. In that interface lurks some fragile assumptions about the number of samples to generate anyways which would get removed as part of this clean up.

brettviren commented 6 years ago

This is wrong thinking. I confused myself about the properties of the Fourier transform.

The noise generation is still too slow but this won't fix it without degrading its validity.

I'll leave this issue open. Maybe someone has an idea.

brettviren commented 6 years ago

Starting time ~16 seconds to get the noise frame (on hierocles workstation).

$ ./build/gen/test_noisesource 
...
TICK: 0 ms (this: 0 ms) start
TICK: 24 ms (this: 24 ms) configuration done
TICK: 16292 ms (this: 16267 ms) got noise frame
TICK: 19257 ms (this: 2964 ms) filled histogram
TICK: 63031 ms (this: 43774 ms) closed ROOT file
brettviren commented 6 years ago

With google perftools

$ CPUPROFILE=noise.prof LD_PRELOAD=~/.nix-profile/lib/libprofiler.so ./build/gen/test_noisesource 
$ google-pprof --gif `which wire-cell` noise.prof  > noise.gif

noise

Ignore the ROOT I/O branch. As expected, the bulk of time is spent drawing randoms and in particular, Gaussian randoms.

brettviren commented 6 years ago

Xin made factor of 2 speed up.