gnuradio / volk

The Vector Optimized Library of Kernels
http://libvolk.org
GNU Lesser General Public License v3.0
557 stars 202 forks source link

Allow for rounding error in float-to-int interleave #718

Closed argilo closed 11 months ago

argilo commented 11 months ago

When building in the ASAN configuration on i386, the volk_32f_x2_s32f_interleave_16ic and qa_volk_32fc_s32f_deinterleave_real_16i tests fail due to integer rounding error:

RUN_VOLK_TESTS: volk_32f_x2_s32f_interleave_16ic(131071,1)
a_sse2 completed in 0.194501 ms
a_sse completed in 2.05456 ms
generic completed in 2.39459 ms
offset 176115 in1: -65 in2: -64 tolerance was: 0
volk_32f_x2_s32f_interleave_16ic: fail on arch a_sse2
offset 176115 in1: -65 in2: -64 tolerance was: 0
volk_32f_x2_s32f_interleave_16ic: fail on arch a_sse
RUN_VOLK_TESTS: volk_32fc_s32f_deinterleave_real_16i(131071,1)
a_sse completed in 0.970632 ms
generic completed in 1.07902 ms
offset 45409 in1: 307 in2: 306 tolerance was: 0
offset 54423 in1: -263 in2: -262 tolerance was: 0
offset 108811 in1: 229 in2: 230 tolerance was: 0
volk_32fc_s32f_deinterleave_real_16i: fail on arch a_sse

To fix this, I've increased their tolerances to 1, after which the tests pass reliably.