gnuradio / volk

The Vector Optimized Library of Kernels
http://libvolk.org
GNU Lesser General Public License v3.0
557 stars 202 forks source link

Fix truncate-toward-zero distortion #709

Closed argilo closed 12 months ago

argilo commented 1 year ago

Fixes #673.

Most kernels with integer outputs use rintf to round the output. But volk_32f_x2_dot_prod_16i and volk_32fc_s32f_deinterleave_real_16i cast directly to int16_t, which introduces truncate-toward-zero distortion. Here I've switched the affected kernels to use rintf instead.

argilo commented 1 year ago

GNU Radio uses volk_32f_x2_dot_prod_16i in the Float->Short variants of the Decimating & Interpolating FIR Filter blocks. Distortion can be observed with the following flow graph: Screenshot from 2023-11-14 13-26-17

Before: Screenshot from 2023-11-14 13-26-27

After: Screenshot from 2023-11-14 13-35-46

argilo commented 1 year ago

It's a bit easier to see what's going on with the floating point signal in the picture as well:

Screenshot from 2023-11-14 20-32-55

Before:

Screenshot from 2023-11-14 20-32-21

After:

Screenshot from 2023-11-14 20-27-46

argilo commented 12 months ago

It turns out there are two GNU Radio tests which assert the truncate-toward-zero behaviour:

https://github.com/gnuradio/gnuradio/blob/9b22fd38b8259f33928c7b3ff7868e0bc4b60d1e/gr-filter/python/filter/qa_fir_filter.py#L160-L188

Those tests now fail. I'll have to think about what the best solution is.

argilo commented 12 months ago

The failing tests were fixed in https://github.com/gnuradio/gnuradio/pull/6980.