sfilip / fquantizer

Euclidean lattice-based quantization routines for digital filter design
GNU General Public License v3.0
1 stars 1 forks source link

fquantizer patch for excessive memory consumption #4

Open jlwehle opened 5 years ago

jlwehle commented 5 years ago

This is simply a port of a portition of a patch I filed for firpm.

pm.cpp has lambdas of the form:

freqBands[i].amplitude = [=](BandSpace bSpace, ...

Unfortunately the [=] appears to cause the a and freqBands arrays to be copied over and over. The net effect is the process consumed 64 GBytes of memory before dying.

-- John PatchJLW03.txt