vectorclass / version2

Vector class library, latest version
Apache License 2.0
1.3k stars 148 forks source link

sin / cos return nan for large negative values instead of 0 #40

Closed janm31415 closed 4 years ago

janm31415 commented 4 years ago

Dear, the following examples generate nan, but (if I'm not mistaken) should return 0 according to the documentation (I used Visual Studio 2017) :

__m256 x = _mm256_set1_ps(-277076958175912885225521152.f);
Vec8f vx(x);
Vec8f p = sin(vx);

__m256  x = _mm256_set1_ps(std::numeric_limits<float>::infinity());
Vec8f vx(x);
Vec8f p = sin(vx);

__m256  x = _mm256_set1_ps(-std::numeric_limits<float>::infinity());
Vec8f vx(x);
Vec8f p = sin(vx);

I think the issue can be fixed by changing lines 228 and 229 in file vectormath_trig.h to

overflow = BVTYPE(q > 0x2000000 | q < 0);  // q big but also check for the sign bit
overflow |= ~is_finite(xa); // set overflow if xa is infinite

Kind regards.

AgnerF commented 4 years ago

Thank you. It is fixed now in https://github.com/vectorclass/version2 INF should generate NAN according to the IEEE-754 standard. Now sin(big) = 0. tan(big) = 0. cos(big) = 1. INF input gives NAN.