cmath test fails on Solaris 10

[5996bd1d-e940-4043-9218-45ba363184f6]

BPO	3168
Nosy	@terryjreedy, @mdickinson
Files	issue3168.patch: Output diagnostic info in cmath_log SunC-64bit-xO5-bug.c: Sun C -xO5 optimization bug on Opteron. config.log.gz: sample Python 2.6b1 SUN C build config.log file config64.log.gz: sample Python 2.6b1 64-bit SUN C config.log

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = 'https://github.com/mdickinson' closed_at = created_at = labels = ['type-bug', 'tests'] title = 'cmath test fails on Solaris 10' updated_at = user = 'https://bugs.python.org/MrJean1' ``` bugs.python.org fields: ```python activity = actor = 'MrJean1' assignee = 'mark.dickinson' closed = True closed_date = closer = 'mark.dickinson' components = ['Tests'] creation = creator = 'MrJean1' dependencies = [] files = ['10763', '10775', '10817', '10818'] hgrepos = [] issue_num = 3168 keywords = ['patch'] message_count = 23.0 messages = ['68549', '68838', '68854', '68864', '68869', '68881', '68883', '68884', '68887', '68889', '68894', '68899', '68900', '68902', '68955', '68957', '68958', '68961', '68963', '69269', '69282', '69291', '69297'] nosy_count = 3.0 nosy_names = ['terry.reedy', 'mark.dickinson', 'MrJean1'] pr_nums = [] priority = 'high' resolution = 'fixed' stage = None status = 'closed' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue3168' versions = ['Python 2.6'] ```

[5996bd1d-e940-4043-9218-45ba363184f6]

There is on cmath test failure with 64-bit Python 2.6b1 and 3.0b1 on Solaris 10. The failure does not occur in the 32-bit builds.

All builds are compiled with Sun's C compiler using the same options (except -xtarget=native vs -xtarget=native64).

\====================================================================== FAIL: test_cmath_matches_math (main.CMathTests) ----------------------------------------------------------------------

Traceback (most recent call last):
  File "Lib/test/test_cmath.py", line 294, in test_cmath_matches_math
    self.rAssertAlmostEqual(math.log(v, base), z.real)
  File "Lib/test/test_cmath.py", line 128, in rAssertAlmostEqual
    self.fail("%s and %s are not sufficiently close" % (repr(a), 
repr(b)))
AssertionError: 6.6438561897747244 and 0.71244141339823108 are not 
sufficiently close

[terryjreedy]

Nasty. Here is the test extracted from test/test_cmath.py

import math, cmath

test_values = [0.01, 0.1, 0.2, 0.5, 0.9, 0.99]
positive = test_values + [1.] + [1./x for x in test_values]
for base in [0.5, 2., 10.]:
    for v in positive:
        z = cmath.log(v, base)
        x = math.log(v,base)
        print(base, v, z, x, z.real-x)

On Winxp 3.0b1, |difference| is usually 0.0, else \< 2.8e-17 6.6438561897747244 is, for instance, log2(100). It is also the first pair tested: log(.01, base=.5) 0.7124414133982310 is not close to any valid test output.

On your system, is cmath64.log totally broken or just for base \< 1?

[mdickinson]

This one's quite baffling.

Jean, can you confirm whether cmath.log(0.01, 0.5) and cmath.log(100., 2.0) give the correct answer or not? They should both give

(6.6438561897747244-0j)

(the sign on the imaginary part might turn out as + instead of -; that's okay.)

[5996bd1d-e940-4043-9218-45ba363184f6]

3 different Python 2.6b1 builds:

1) 64-bit Python 2.6b1 on Solaris 10 built with SUN C (no -xlibmieee):

Python 2.6b1 (r26b1:64398, Jun 19 2008, 20:27:39) [C] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> import cmath
>>> cmath.log(0.01, 0.5)
(0.71244141339823108+2.0556715512777863j)
>>> cmath.log(100.0, 2.0)
(0.71244151439608006-2.0556716794852954j)

2) 32-bit Python 2.6b1 on Solaris 10 built with SUN C -xlibmieee:

Python 2.6b1 (r26b1:64398, Jun 24 2008, 13:50:09) [C] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> import cmath
>>> cmath.log(0.01, 0.5)
(6.6438561897747244-0j)
>>> cmath.log(100.0, 2.0)  
(6.6438561897747253+0j)

3) 32-bit Python 2.6b1 on MacOS X 10.4.11 (Intel):

Python 2.6b1 (r26b1:64398, Jun 23 2008, 18:36:08) 
[GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import cmath
>>> cmath.log(0.01, 0.5)
(6.6438561897747244-0j)
>>> cmath.log(100.0, 2.0)   
(6.6438561897747253+0j)

[mdickinson]

More curious and more curious...

I assume that cmath.log(100.0) and cmath.log(2.0) return the right things? (Both answers should have zero imaginary part.)

Might this be some optimization bug? Can you turn off all optimizations and see what happens?

[5996bd1d-e940-4043-9218-45ba363184f6]

For 32-bit, see the latest posts at \http://bugs.python.org/issue3167\.

[mdickinson]

What about cmath.log(100.0) and cmath.log(2.0) on 64-bit?

[5996bd1d-e940-4043-9218-45ba363184f6]

Here are the 64-bit results. First with -xO5:

Python 2.6b1 (r26b1:64398, Jun 28 2008, 10:57:27) [C] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> import cmath
>>> cmath.log(100.0)
(4.6051701859880918+0j)
>>> cmath.log(2.0)
(0.69314718055994529+0j)
>>> cmath.log(100.0, 2.0)
(0.71244151439608006-2.0556716794852954j)
>>> import math
>>> math.log(float('-inf'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: math domain error

This is 64-bit with -xO0:

Python 2.6b1 (r26b1:64398, Jun 28 2008, 11:02:57) [C] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> import cmath
>>> cmath.log(100.0)
(4.6051701859880918+0j)
>>> cmath.log(2.0)
(0.69314718055994529+0j)
>>> cmath.log(100.0, 2.0)
(6.6438561897747253+0j)

>>> import math
>>> math.log(float('-inf'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: math domain error

[5996bd1d-e940-4043-9218-45ba363184f6]

Forgot to mention, both 64-bit builds above includes -xlibmieee in the BASECFLAGS and LDFLAGS ./configure symbols (not LDSHARED!).

The complete .configure command line was (with OPT adjusted accordingly):

env CCSHARED="-KPIC" LDSHARED="cc -xtarget=native64 -G" LDFLAGS="- xlibmieee -xtarget=native64" CC="cc" CPP="cc -xtarget=native64 -E" BASECFLAGS="-xtarget=native64 -xlibmieee" CFLAGS="-xtarget=native64"
CXX="CC -xtarget=native64" OPT="-xO5" ./configure --enable-shared -- without-gcc --disable-ipv6 --prefix=....

[mdickinson]

This looks a lot like a compiler bug, then. Do you agree?

I guess the next step would be to try to extract a smallest failing example from the Python code, and report the bug to Sun. It might first be interesting to insert some printfs in cmath_log to see where things go wrong (cmath_log does 2 log computations and a complex division; any of these 3 operations might be doing something strange. Of course, it's always possible that the compiler bug goes away when the printfs are added.)

But at this point I'm going to claim that this bug isn't Python's fault, and suggest that this issue should be closed.

[5996bd1d-e940-4043-9218-45ba363184f6]

This is pretty bizarre, take a look at the following, 64-bit Pyhon 2.6b1 -xO5.

Python 2.6b1 (r26b1:64398, Jun 28 2008, 12:50:06) [C] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> import math
>>> math.log(100.0, 2.0)
6.6438561897747253
>>> math.log(100.0) / math.log(2.0)
6.6438561897747253

>>> import cmath
>>> cmath.log(100.0, 2.0)
(0.71244151439608006-2.0556716794852954j)
>>> cmath.log(complex(100.0, 0), complex(2.0, 0)) 
(0.71244151439608006-2.0556716794852954j)

>>> cmath.log(2.0)
(0.69314718055994529+0j)
>>> cmath.log(100.0)
(4.6051701859880918+0j)
>>> cmath.log(100.0) / cmath.log(2.0)
(6.6438561897747253+0j)

There is only an issue with cmath.log(100.0, 2.0) but not with cmath.log(100.0) / cmath.log(2.0). That seems to indicate that the c_quot function may be the root cause of the problem.

[5996bd1d-e940-4043-9218-45ba363184f6]

Trying to isolate and duplicate the problem with c_quot isn't quite working. See the attached c_quot.c file and the results at the end.

[mdickinson]

Could you try the attached patch, and see what output you get from cmath.log(100., 2.)? (On 64-bit, of course.)

[mdickinson]

Trying to isolate and duplicate the problem with c_quot isn't quite working. See the attached c_quot.c file and the results at the end.

I'd expect that you'd need both the cmath_log and the c_quot functions, probably in separate files that are compiled independently and then linked together. It looks like it's somehow the combination of these two that's causing a problem.

[5996bd1d-e940-4043-9218-45ba363184f6]

I have not yet been able to duplicate the problem in a smaller test case, but I did stumble into a fix. Changing the following statement in the cmath_log function from

if (PyTuple_GET_SIZE(args) == 2)
    x = c_quot(x, c_log(y));

to

    if (PyTuple_GET_SIZE(args) == 2) {
        y = c_log(y);
        x = c_quot(x, y);
    }

fixes the problem for the 64-bit -xO5 Python build. The result is no the same in all 32- and 64-bit builds with -xO0 and -xO5 on Solaris 10 (Opteron) with SUN C.

>>> cmath.log(100, 2)
(6.6438561897747253+0j)

>>> cmath.log(0.01, 0.5)
(6.6438561897747244-0j)

[5996bd1d-e940-4043-9218-45ba363184f6]

Attached is a test case which does demonstrate the problem. See the top of that file. I will post this at the SUN C Forum.

[5996bd1d-e940-4043-9218-45ba363184f6]

File but showing -xO0 thru -xO5 results.

[5996bd1d-e940-4043-9218-45ba363184f6]

Without changing the cmath_log code, another workaround is to compile Python with optimization level -xO2 or less for 64-bit using Sun C.

[5996bd1d-e940-4043-9218-45ba363184f6]

There is another, perhaps related issue on Solaris. The compiler warns that function finite is implicitly defined.

Commenting out this line in pyconfig.h as

/ #define HAVE_FINITE 1 \/

make that warning go away. If there is no function finite, why is HAVE_FINITE defined at all?

[5996bd1d-e940-4043-9218-45ba363184f6]

Below is the reply from SUN. It was indeed a bug which has been fixed already.

I have not yet applied the patches to my SUN compilers and tried again.

/Jean

Begin forwarded message:

From: Sun Microsystems \IncidentUpdateDaemon@sun.com\ Date: July 1, 2008 3:51:17 PM PDT

Subject: Re: (Incident Review ID: 1284413) Sun C 5.8 optimization bug for 64-bit Opteron

--- Note: you can send us updates about your Incident --- --- by replying to this mail. Place new information --- --- above these lines. Do not include attachments. --- --- Our system ignores attachments and anything below --- --- these lines. ---

Hi Jean Brouwers,

We appreciate your feedback. Using your sample I was able to get the correct results using Sun Studio 12. This may indicate that the issue was also fixed in a backported patch to Studio 11 (Sun C 5.8). Check to be sure that the latest patches have been applied to your installation. See:

\http://developers.sun.com/sunstudio/downloads/patches/ss11_patches.html\

Regards, Brad Mayer

\~~~~~~~~~~~~~~~~ NOTICE: This message, including any attachments, is for the intended recipient(s) only. If you are not the intended recipient(s), please reply to the sender, delete this message, and refrain from disclosing, copying, or distributing this message. \~~~~~~~~~~~~~~~~

--------------- Previous Messages ----------------

--------------------- Report ---------------------

  category : c

subcategory : compiler release : other type : bug synopsis : Sun C 5.8 optimization bug for 64-bit Opteron customer name : Jean Brouwers sdn id : language : en hardware : x86 os : sol2.5.1 bug id : 0 date created : Sun Jun 29 11:49:10 MST 2008 date evaluated : Tue Jul 01 15:41:10 MST 2008 description :

FULL PRODUCT VERSION : which cc /opt/SUNWspro/bin/cc cc -V cc: Sun C 5.8 2005/10/13

ADDITIONAL OS VERSION INFORMATION : uname -a SunOS unknown 5.10 Generic_118855-14 i86pc i386 i86pc

EXTRA RELEVANT SYSTEM CONFIGURATION : Solaris 10 on an Ultra20 Opteron machine.

A DESCRIPTION OF THE PROBLEM : At higher optimization levels, the Sun C compiler does not handle certain function call patterns correctly. Specifically, the values of complex_t variables q and r in the code snippet below are different.

typedef struct {
    double real;
    double imag;
} complex_t;

complex_t c_comp(double real); complex_t c_quot(complex_t a, complex_t b); ... complex_t_t x, y, q, r;

    x = c_comp(4.6051701859880918);
    y = c_comp(0.69314718055994529);

    q = c_quot(x, y);

    r = c_quot(x, c_comp(0.6931471805599452));
...

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM : See the attached test case. Instructions are included inside.

EXPECTED VERSUS ACTUAL BEHAVIOR : EXPECTED -

rm a.out ; cc -xtarget=native64 -xO0 c_main.c c_quot.c -lm ; ./a.out c_main.c: c_quot.c: 6.643856 + j 0.000000 6.643856 + j 0.000000

ACTUAL -

rm a.out ; cc -xtarget=native64 -xO3 c_main.c c_quot.c -lm ; ./a.out c_main.c: c_quot.c: 6.643856 + j 0.000000 Inf + j 0.000000

REPRODUCIBILITY : This bug can be reproduced always.

---------- BEGIN SOURCE ----------

/* Split this file in 3 separate file, c_main.c, c_quot.h and c_quot.c. Compile with and without optimization using Sun C 5.8 compiler on Solaris 10 (Opteron) and run as follows.

rm a.out ; cc -xtarget=native64 -xO0 c_main.c c_quot.c -lm ; ./a.out c_main.c: c_quot.c: 6.643856 + j 0.000000 6.643856 + j 0.000000

rm a.out ; cc -xtarget=native64 -xO1 c_main.c c_quot.c -lm ; ./a.out c_main.c: c_quot.c: 6.643856 + j 0.000000 6.643856 + j 0.000000

rm a.out ; cc -xtarget=native64 -xO2 c_main.c c_quot.c -lm ; ./a.out c_main.c: c_quot.c: 6.643856 + j 0.000000 6.643856 + j 0.000000

rm a.out ; cc -xtarget=native64 -xO3 c_main.c c_quot.c -lm ; ./a.out c_main.c: c_quot.c: 6.643856 + j 0.000000 Inf + j 0.000000

rm a.out ; cc -xtarget=native64 -xO4 c_main.c c_quot.c -lm ; ./a.out c_main.c: c_quot.c: 6.643856 + j 0.000000 Inf + j 0.000000

rm a.out ; cc -xtarget=native64 -xO5 c_main.c c_quot.c -lm ; ./a.out c_main.c: c_quot.c: 6.643856 + j 0.000000 Inf + j 0.000000

*/

------------------ save as c_main.c -----------------

#include <stdio.h>
#include "c_quot.h"

int main(int argc, char* argv[])
{
    complex_t x, y, q, r;

    x = c_comp(4.6051701859880918);
    y = c_comp(0.69314718055994529);
    q = c_quot(x, y);
    /* expect: 6.643856 + j 0.000000 */
    printf("%f + j %f\n", q.real, q.imag);

    x = c_comp(4.6051701859880918);
    y = c_comp(0.69314718055994529);
    r = c_quot(x, c_comp(0.6931471805599452));
    /* expect: 6.643856 + j 0.000000, but ... */
    printf("%f + j %f\n", r.real, r.imag);
}

------------------ save as c_quot.h -----------------

typedef struct {
    double real;
    double imag;
} complex_t;

complex_t c_comp(double real); complex_t c_quot(complex_t a, complex_t b);

------------------ save as c_quot.c -----------------

#include "c_quot.h"

complex_t
c_comp(double real)
{
    complex_t c;
    c.real = real;
    c.imag = 0.0; /* ignore imag */
    return c;
}

complex_t
c_quot(complex_t a, complex_t b)
{
    complex_t r;
    r.real = a.real / b.real;
    r.imag = 0.0; /* ignore imag */
    return r;
}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND : Compiling with -xO... less than 3 avoids the problem.

[mdickinson]

Thanks, Jean. I've checked in your workaround in r64735.

Leaving this open for now as a reminder about finite/is_finite.

[mdickinson]

There is another, perhaps related issue on Solaris. The compiler warns that function finite is implicitly defined.

Commenting out this line in pyconfig.h as

/ #define HAVE_FINITE 1 \/

make that warning go away. If there is no function finite, why is HAVE_FINITE defined at all?

I don't think this is too serious. My guess would be that both finite and isfinite (which is the C99-recommended way to spell finite) are implemented in libm, but that only isfinite is mentioned in math.h (or possibly that finite is mentioned is math.h but is ifdef'd out as a result of some particular compiler options).

So a code snippet that refers to 'finite' emits a warning, but compiles fine. Hence the corresponding autoconf test passes, and autoconf sets HAVE_FINITE to 1. And the Python build also emits a warning, but compiles and runs fine.

Could you take a look in the 'config.log' file after configuring and see whether the 'implicit definition of finite' warning is in there as well?

The right fix is probably to rework things to use 'isfinite' in preference to 'finite'. I'm not sure that Windows has isfinite, though.

[5996bd1d-e940-4043-9218-45ba363184f6]

Can't find that particular message. Attached is the entire log of one of the SUN C builds of Python 2.6b1.