Open Crotchfire opened 5 years ago
I believe the most crucial problem I'm seeing comes from line 20 in "sqrt_v2_single_precision.h":
__m128 dx = _mm_cvtsi64_ss(z, static_cast(n0) - x0 * x0);
as n0 is the unbiased input and x0 is the square root after biasing the input, so the resulting value doesn't accurately represent the difference between the input and the current estimate of the sqrt squared.
I believe the most crucial problem I'm seeing comes from line 20 in "sqrt_v2_single_precision.h":
__m128 dx = _mm_cvtsi64_ss(z, static_cast(n0) - x0 * x0);
as n0 is the unbiased input and x0 is the square root after biasing the input, so the resulting value doesn't accurately represent the difference between the input and the current estimate of the sqrt squared.