New module complex_mpc using lib mpc for complex multiprecision arithmetic

459712a1-0454-42be-81de-d594de9e5edf commented 16 years ago

Herewith and there (http://www.loria.fr/~thevenyp/complex_mpc.patch 38K) is a patch with new classes using the MPC library for complex multi-precision arithmetic (see ticket #4308 for the associated spackage).

This is an adaptation of the module real_mpfr and of ComplexField and ComplexNumber classes. It adds a class MPComplexField with precision (common to both part) and rounding modes (specific to each part) and a class MPComplexNumber.

This first attempt implements only the complex arithmetic.

The test suite does fail due to coercion problems I can't solve.

CC: @robertwb @williamstein @mwhansen @sagetrac-mvngu @haraldschilly @nexttime

Component: basic arithmetic

Author: Philippe Theveny, Alex Ghitza, Yann Laigle-Chapuy

Reviewer: William Stein, David Kirkby, Paul Zimmermann, Mitesh Patel

Merged: sage-4.6.alpha2

Issue created by migration from https://trac.sagemath.org/ticket/4446

459712a1-0454-42be-81de-d594de9e5edf commented 16 years ago

Attachment: complex_mpc.patch.gz

aghitza commented 15 years ago

Attachment: trac4446-complex_mpc.patch.gz

replace the previous patch (rebase against 3.2)

aghitza commented 15 years ago

comment:2

There seems to have been some bitrot due to recent changes in Sage. I have uploaded a very slightly modified version of Philippe's patch, which applies cleanly against 3.2. A number of doctests fail for various reasons. I will try to look into this soon.

459712a1-0454-42be-81de-d594de9e5edf commented 15 years ago

Attachment: complex_mpc.p0.patch.gz

coercion problems resolved, more functions

459712a1-0454-42be-81de-d594de9e5edf commented 15 years ago

comment:3

The coercion problems have been solved, all doctests now succeed in new version. The whole set of mpc functions involving complex numbers is now in the interface.

aghitza commented 15 years ago

apply after complex_mpc.p0.patch

aghitza commented 15 years ago

comment:4

Attachment: trac4446_fix.patch.gz

Philippe,

Great work! I'll do my best to review this for 3.2.2. Right now I notice that the patch doesn't work in 3.2.1 because of #4580. I'm attaching a tiny patch that fixes that (should be applied after complex_mpc.p0.patch.

aghitza commented 15 years ago

comment:5

There are a few small issues that I ran into, but since this is going to be (for now) an optional part of Sage and nothing depends on it, I think we can fix these later (I'm keeping a list of them so they don't get lost).

However, there is a policy of 100% doctest for all new code. The coverage is now:

[ghitza@artin sage]$ sage -coverage rings/complex_mpc.pyx
----------------------------------------------------------------------
rings/complex_mpc.pyx
SCORE rings/complex_mpc.pyx: 68% (46 of 67)

Missing documentation:
     * _repr_ (self):
     * _latex_(self):
     * _an_element_(self):
     * random_element(self, bound):
     * __hash__(self):
     * prec(self):
     * _set(self, z, y=None, base=10):
     * _repr_(self):
     * _latex_(self):
     * ModuleElement _add_(self, ModuleElement right):
     * ModuleElement _sub_(self, ModuleElement right):
     * RingElement _mul_(self, RingElement right):
     * RingElement _div_(self, RingElement right):
     * ModuleElement _neg_(self):
     * __create__MPComplexField_version0 (prec, rnd):
     * __create_MPComplexNumber_version0 (parent, s, base=10):

Missing doctests:
     * _rnd(self):
     * _rnd_re(self):
     * _rnd_im(self):
     * _real_field(self):
     * _imag_field(self):

Possibly wrong (function name doesn't occur in doctests):
     * _element_constructor_(self, z):
     * _coerce_map_from_(self, S):
     * bint is_exact(self) except -2: return False def is_finite(self):
     * Element _call_(self, z):
     * Element _call_(self, x):

----------------------------------------------------------------------

So the missing docstrings and doctests will have to be added before this patch can be merged. I would happily do this but the earliest I can get to it is Thursday or so (and that's being optimistic).

To end on a positive note: this looks good. It will take a bit more work, but if we can show that (a) the performance is improved, or doesn't suffer too much, and (b) the switch can be made seamlessly, then it will be easy to convince people that we should switch the core of our complex numbers functionality over to MPC.

aghitza commented 15 years ago

apply after complex_mpc.p0.patch and trac4446_fix.patch

aghitza commented 15 years ago

comment:7

Attachment: trac4446_doctests.patch.gz

I added a patch trac4446_doctests.patch, which does a number of things:

adds doctests for all functions except three internal use only functions
changes _repr_ of complex numbers so that it agrees with the way complex numbers are currently printed in Sage
makes MPComplexField inherit from ParentWithGens, being generated over its real field by the square root of -1 (just as it is now); this required adding a few functions. So now one can do

sage: from sage.rings.complex_mpc import MPComplexField
sage: MPC.<j> = MPComplexField()
sage: j^2
-1.00000000000000 + 0.000000000000000*I

Note that only the last three patches should be applied, in order: complex_mpc.p0.patch, trac4446_fix.patch, and trac4446_doctests.patch

459712a1-0454-42be-81de-d594de9e5edf commented 15 years ago

Attachment: trac4446_norm.patch.gz

459712a1-0454-42be-81de-d594de9e5edf commented 15 years ago

comment:8

One more patch trac4446_norm.patch with:

Alex Ghitza listed in authors list and copyright notice
complex_mpc uses a copy of mpfr rounding mode list instead of its private one
the __abs__ and norm documentation has been improved.

williamstein commented 15 years ago

comment:9

REFEREE REPORT:

How can this be optional? If the patch is applied then this is included:

    625     Extension('sage.rings.complex_mpc', 
    626               sources = ['sage/rings/complex_mpc.pyx'], 
    627               libraries = ['mpc', 'mpfr', 'gmp']), \ 
    628

So the only way I can see this going in like this is if it is standard. Am I missing something (probably)?

Timings on Xeon 2.6Ghz (sage.math). Multiplication is faster with mpc, but addition is significantly slower:

sage: K = MPComplexField(53)
sage: a = K(3.3902384)
sage: b = CC(3.3902384)

sage: timeit('a*a')
625 loops, best of 3: 376 ns per loop
sage: timeit('b*b')
625 loops, best of 3: 466 ns per loop

sage: timeit('a+a')
625 loops, best of 3: 368 ns per loop
sage: timeit('b+b')
625 loops, best of 3: 304 ns per loop

Powering doesn't work for mpc but it does for the current CC:

sage: a**a 
boom

Default rounding on coercion (or something?!) is different between MPC and CC:

sage: MPComplexField(100)(3.3902384)
3.3902383999999998742680418218 + 0.00000000000000000000000000000*I
sage: ComplexField(100)(3.3902384)
3.3902384000000000000000000000

With higher precision and nontrivial imaginary part, the timings are all in favor of the existing ComplexField already in Sage:

sage: K = MPComplexField(1000)
sage: a = K(3.3902384,9203483)
sage: b = ComplexField(1000)(3.3902384,9203483)
sage: a
3.39023839999999987426804182177875190973281860351562500000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 + 9.20348300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e6*I
sage: b
3.39023840000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 + 9.20348300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e6*I
sage: timeit('a*a')
625 loops, best of 3: 2.27 µs per loop
sage: timeit('b*b')
625 loops, best of 3: 2.2 µs per loop
sage: timeit('a+a')
625 loops, best of 3: 515 ns per loop
sage: timeit('b+b')
625 loops, best of 3: 445 ns per loop
sage: timeit('a.sin()')
^P625 loops, best of 3: 221 µs per loop
sage: timeit('b.sin()')
625 loops, best of 3: 130 µs per loop

Thus until the above issues are addressed, I see no point in mpc going into sage.

zimmermann6 commented 15 years ago

Attachment: complex_mpc.pxd.gz

zimmermann6 commented 15 years ago

Attachment: complex_mpc.pyx.gz

zimmermann6 commented 15 years ago

Attachment: mpc.patch.gz

Attachment: mpc.pxi.gz

zimmermann6 commented 15 years ago

comment:10

Philippe Theveny, who had a 2-year contract with us, just left, thus he won't be able to continue working on this. In case somebody wants to pursue his work, he has left the attached files. You need mpc-0.6 available at http://www.loria.fr/~thevenyp/mpc-0.6.spkg. According to Philippe, the performance was improved, but still not as good as the current CC (but more reliable due to correct rounding).

2c00b582-cfe9-43b9-8124-fec470060704 commented 14 years ago

Attachment: mpc-0.8.1.patch.gz

2c00b582-cfe9-43b9-8124-fec470060704 commented 14 years ago

comment:11

With the updated patch, and the corresponding spkg available at http://yann.laiglechapuy.net/spkg/mpc-0.8.1.p0.spkg almost every methods for ComplexNumbers are defined for MPComplexNumbers to.

The performance is quite good: with a precision of 500bits,

==========
add
ComplexField: a+b 40000 loops, best of 5: 700 ns per loop
MPComplexField: m+n 40000 loops, best of 5: 755 ns per loop
==========
add int
ComplexField: a+17 40000 loops, best of 5: 2.75 µs per loop
MPComplexField: m+17 40000 loops, best of 5: 2.43 µs per loop
==========
mul
ComplexField: a*b 40000 loops, best of 5: 2.66 µs per loop
MPComplexField: m*n 40000 loops, best of 5: 2.68 µs per loop
==========
div
ComplexField: a/b 40000 loops, best of 5: 5.76 µs per loop
MPComplexField: m/n 40000 loops, best of 5: 10.7 µs per loop
==========
real
ComplexField: a.real() 40000 loops, best of 5: 1.7 µs per loop
MPComplexField: m.real() 40000 loops, best of 5: 764 ns per loop
==========
conj
ComplexField: a.conjugate() 40000 loops, best of 5: 756 ns per loop
MPComplexField: m.conjugate() 40000 loops, best of 5: 633 ns per loop
==========
arg
ComplexField: a.argument() 2000 loops, best of 5: 104 µs per loop
MPComplexField: m.argument() 2000 loops, best of 5: 102 µs per loop
==========
cos
ComplexField: a.cos() 2000 loops, best of 5: 57.2 µs per loop
MPComplexField: m.cos() 2000 loops, best of 5: 86.4 µs per loop
==========
pow
ComplexField: a**b 2000 loops, best of 5: 240 µs per loop
MPComplexField: m**n 2000 loops, best of 5: 515 µs per loop
==========
pow int
ComplexField: a**12345 2000 loops, best of 5: 55.1 µs per loop
MPComplexField: m**12345 2000 loops, best of 5: 549 µs per loop
==========
log
ComplexField: a.log() 4000 loops, best of 5: 177 µs per loop
MPComplexField: m.log() 4000 loops, best of 5: 153 µs per loop
==========
exp
ComplexField: a.exp() 10000 loops, best of 5: 53.7 µs per loop
MPComplexField: m.exp() 10000 loops, best of 5: 57.5 µs per loop

The only big slow down is for pow.

It also allows to compute various trigonometric functions without convertion to pari.

2c00b582-cfe9-43b9-8124-fec470060704 commented 14 years ago

comment:12

Erratum: The only big slow down are for pow and div

2c00b582-cfe9-43b9-8124-fec470060704 commented 14 years ago

apply after mpc-0.8.1.patch

2c00b582-cfe9-43b9-8124-fec470060704 commented 14 years ago

comment:13

Attachment: bug_literal.patch.gz

I put it as needs review to know what has eventually to be done.

bac7d3ea-3f1b-4826-8464-f0b53d5e12d2 commented 14 years ago

comment:15

Has this been checked on Solaris? There have been issues with complex maths recently on Solaris, which were solved, but this would need testing.

There's general information about building on Solaris at

http://wiki.sagemath.org/solaris

Information specifically for 't2' at

http://wiki.sagemath.org/devel/Building-Sage-on-the-T5240-t2

Both the source (4.3.0.1 is the latest to build on Solaris) and a binary which will run on any SPARC can be found at http://www.sagemath.org/download-source.html

Dave

34e476cc-07ba-4e3a-bc99-20f04bc433a9 commented 14 years ago

comment:16

The poor performance of "pow int" is due to the fact that binary exponentiation is not currently implemented in mpc; "pow int" just calls "pow", and is present as a convenience function. Binary exponentiation is on the todo list.

Except for special cases, "pow" uses log and exp, so the timing should essentially be the sum for these two functions. Is it possible that you hit a corner case with your experiments, or are the timings consistent over several arguments?

Similarly, what are the arguments for division?

Concerning solaris, mpc itself builds without problems, see http://www.multiprecision.org/index.php?prog=mpc&page=platforms

zimmermann6 commented 14 years ago

comment:17

Except for special cases, "pow" uses log and exp, so the timing should essentially be the sum for these two functions. Is it possible that you hit a corner case with your experiments, or are the timings consistent over several arguments?

then we should get about 210us for powint instead of 549us. Indeed, it would be good to know the exact arguments, so that we can try to reproduce with vanilla MPC (maybe the problem is in the interface between Sage and MPC).

For division, GMP 5 has a much faster division, which might benefit to MPC too. But again, we could investigate more with a vanilla MPC, if we know the exact inputs used (with exact_rational).

2c00b582-cfe9-43b9-8124-fec470060704 commented 14 years ago

Attachment: trac4446-rebase_sage4.3.3.patch.gz

2c00b582-cfe9-43b9-8124-fec470060704 commented 14 years ago

comment:18

First, I added a small patch to rebase this on sage 4.3.3.

One need to apply:

mpc-0.8.1.patch
bug_literal.patch
trac4446-rebase_sage4.3.3.patch

in this order; and of course install the spkg:

http://yann.laiglechapuy.net/spkg/mpc-0.8.1.p0.spkg

Then, here is an example with the exact input (I only put the last part concerning pow)

a.real().exact_rational()
-1558022726154386835635734102276225872461248356241845468520145243680772756232147166398716395960573602794592840676040928991781722240298082039889553874139/1636695303948070935006594848413799576108321023021532394741645684048066898202337277441635046162952078575443342063780035504608628272942696526664263794688
a.imag().exact_rational()
38743143824869548998159480051850490241205026500878413377278464034858416981434423320408166976481329585622042476431430168493334123089110626626927265425/102293456496754433437912178025862473506770063938845774671352855253004181137646079840102190385184504910965208878986252219038039267058918532916516487168
------------------------
a.log()
CC 5000 loops, best of 3: 170 µs per loop
MPC 5000 loops, best of 3: 143 µs per loop
------------------------
a.exp()
CC 5000 loops, best of 3: 45.6 µs per loop
MPC 5000 loops, best of 3: 49.1 µs per loop
------------------------
a**12345
CC 5000 loops, best of 3: 48.7 µs per loop
MPC 5000 loops, best of 3: 501 µs per loop

2c00b582-cfe9-43b9-8124-fec470060704 commented 14 years ago

comment:19

and for div (same value for a)

b.real().exact_rational()
-1335661139895223805455191750131391965080983427340135233479028132544481051566385358748782698880424407717119644643310969153530607585226701160632690491359/1636695303948070935006594848413799576108321023021532394741645684048066898202337277441635046162952078575443342063780035504608628272942696526664263794688
b.imag().exact_rational()
-191023642554886415830676606502854964270751830339806573746974716999454958912475563080101457087388705931975286670167330679505998875813037449880206143447/409173825987017733751648712103449894027080255755383098685411421012016724550584319360408761540738019643860835515945008876152157068235674131666065948672
------------------------
a/b
CC 100000 loops, best of 3: 4.84 µs per loop
MPC 100000 loops, best of 3: 9.57 µs per loop

zimmermann6 commented 14 years ago

comment:20

with the attached MPC program I get on a 2.83Ghz Core 2 under Fedora Core 2, with GMP 5.0.1 and MPFR 2.4.2

tarte% ./sage
mpc_log took 74us per loop
mpc_exp took 24us per loop
mpc_pow(12345) took 260us per loop
mpc_div took 3.4us per loop

and within Sage with ComplexField I get (where I took the best of 3 %timeit runs):

sage: %timeit c=a.log()
625 loops, best of 3: 173 µs per loop
sage: %timeit c=a.exp()
625 loops, best of 3: 37.1 µs per loop
sage: %timeit c=a**12345
625 loops, best of 3: 26.6 µs per loop
sage: %timeit c=a/b
625 loops, best of 3: 2.71 µs per loop

thus we indeed need to work on powint, but the division is only 25% slower.

While doing this test I also noticed this strange thing with ComplexField:

sage: smallb = CC(b)
sage: %timeit c=a/b
625 loops, best of 3: 2.69 µs per loop
sage: %timeit c=a/smallb
625 loops, best of 3: 290 µs per loop

thus dividing by a 53-bit complex is 100 times slower than dividing by a 500-bit complex! Should I open a separate ticket for that?

Did somebody check the 500-bit outputs of CC and MPC agree?

zimmermann6 commented 14 years ago

Attachment: sage.c.gz

zimmermann6 commented 14 years ago

comment:21

By studying what happens with the powint example, there was a failure in Ziv's strategy, thus we did compute two approximations, one with precision 511 bits (which was not enough), and one with a larger precision. By adding one more guard bit, the first approximation with 512 bits is enough to get correct rounding, and we now get 131us per loop instead of 260us (in vanilla MPC). A further gain could be obtained if we get rid of the mpc_log call to detect overflow or underflow: we would then get 113us. But of course the definite solution will be to implement binary exponentiation.

zimmermann6 commented 14 years ago

comment:22

We have now implemented binary exponentiation in mpc_pow_ui. We now get on the same machine as above with the svn version of MPC:

tarte% ./sage
mpc_log took 75us per loop
mpc_exp took 24us per loop
mpc_pow(12345) took 19us per loop
mpc_div took 3.5us per loop

thus now MPC powint should be comparable to CC powint. There is not yet a release of MPC with the new code, but it suffices to modify the file pow_ui.c.

zimmermann6 commented 14 years ago

comment:23

I put it as "needs review" again to know if some more issues need to be fixed upstream. Paul

2c00b582-cfe9-43b9-8124-fec470060704 commented 14 years ago

comment:24

Replying to @zimmermann6:

I put it as "needs review" again to know if some more issues need to be fixed upstream. Paul

If you send me the file pow_ui.c I can patch my spkg so that people can try it. Yann

zimmermann6 commented 14 years ago

comment:25

If you send me the file pow_ui.c I can patch my spkg so that people can try it. Yann

sorry, it is available from https://gforge.inria.fr/scm/viewvc.php/trunk/src/pow_ui.c?root=mpc We might still improve it, thus if you update your spkg please indicate the revision of the file you took.

Paul

jasongrout commented 14 years ago

comment:26

Paul,

Now with MPC 0.8.2, mpz_pow_ui is fast, but mpz_pow_si is still slow. As the MPC Sage code is written right now, calls are made to mpz_pow_si, mpz_pow_z, or mpz_pow, depending on the type of argument. Each of these are still 5-6 slower than Sage's complex number type.

Do your great speedups for mpz_pow_ui make it easy to speed up mpz_pow_si, mpz_pow_z, or mpz_pow as well?

jasongrout commented 14 years ago

comment:27

Replying to @jasongrout:

Paul,

Now with MPC 0.8.2, mpz_pow_ui is fast, but mpz_pow_si is still slow. As the MPC Sage code is written right now, calls are made to mpz_pow_si, mpz_pow_z, or mpz_pow, depending on the type of argument. Each of these are still 5-6 slower than Sage's complex number type.

I mean: each of these calls (mpz_pow[_si|_z]) results in operations that are 5-6 times as long as Sage's complex type (~35 microseconds vs. ~200 microseconds).

jasongrout commented 14 years ago

comment:28

Paul, I see in the MPC repository that you are already working on this. Thanks!

zimmermann6 commented 14 years ago

comment:29

Replying to @jasongrout:

Jason,

Paul, I see in the MPC repository that you are already working on this. Thanks!

yes we are working on this. Too bad that you didn't tell us before the 0.8.2 release. Are you subscribed to the mpc-discuss list, where we ask feedback for release candidates?

jasongrout commented 14 years ago

comment:30

Replying to @zimmermann6:

yes we are working on this. Too bad that you didn't tell us before the 0.8.2 release. Are you subscribed to the mpc-discuss list, where we ask feedback for release candidates?

I only just was able to squeeze some time out to look at this. I'm not subscribed to mpc-discuss; I'll subscribe now.

zimmermann6 commented 14 years ago

comment:31

Revision svn 790 from MPC implements a faster mpc_pow_si:

tarte% ./sage
mpc_log took 76us per loop
mpc_exp took 24us per loop
mpc_pow_ui(12345) took 19.2us per loop
mpc_pow_ui(4294967295) took 149.0us per loop
mpc_pow_si(12345) took 19.2us per loop
mpc_pow_si(-12345) took 22.6us per loop
mpc_pow_z(12345) took 19.2us per loop
mpc_pow(12345) took 133.4us per loop
mpc_div took 3.6us per loop

Yann, please can you prepare a new spkg so that Jason can review this ticket?

Paul

bac7d3ea-3f1b-4826-8464-f0b53d5e12d2 commented 14 years ago

comment:32

In the spkg-install I see above:

if [ `uname` = "Darwin" -a "$SAGE64" = "yes" ]; then
   echo "64 bit MacIntel"
   CFLAGS="-O2 -g -m64 -fPIC"; export CFLAGS
   CXXLAGS="-O2 -g -m64 -fPIC"; export CXXFLAGS
   LDFLAGS="-m64 "; export LDFLAGS
else
   CFLAGS="-O2 -g -fPIC"; export CFLAGS
fi

This will cause an issue that plagues a 64-bit port to OpenSolaris and Solaris 10. There is no reason to restrict the -m64 flag to be just OS X. It should be changed to:

if [ "x$SAGE64" = xyes ]; then
   echo "Building a 64-bit version of MPC"
   CFLAGS="-O2 -g -m64 -fPIC"; export CFLAGS
   CXXLAGS="-O2 -g -m64 -fPIC"; export CXXFLAGS
   LDFLAGS="-m64 "; export LDFLAGS
else
   CFLAGS="-O2 -g -fPIC"; export CFLAGS
fi

Was there a good reason for unsetting 'RM' in the spkg-install, or was it just one copied from some other package, which happened to do that?

Most packages use

configure
make
make install

checking the error code of each of the 3 stages. This does

configure
make install

If 'make install' fails, there is no way to know if the build or installation failed.

See

http://www.sagemath.org/doc/developer/producing_spkgs.html#creating-a-new-spkg

for some information on that. I note looking at the above link, there is no code which does anything when SAGE64 is set to "yes". That is in my mind an error in the documentation. Clearly that was written even before a 64-bit OS X port was attempted.

Also, SPKG.txt is an empty file - again see that link above for how that should be created.

Dave

2c00b582-cfe9-43b9-8124-fec470060704 commented 14 years ago

comment:33

new spkg here:

http://yann.laiglechapuy.net/spkg/mpc-0.8.2svn793.spkg

2c00b582-cfe9-43b9-8124-fec470060704 commented 14 years ago

comment:34

Attachment: fix_declaration.patch.gz

after installing 'mpc-0.8.2svn793.spkg' , one needs to apply in order:

mpc-0.8.1.patch
bug_literal.patch
trac4446-rebase_sage4.3.3.patch
fix_declaration.patch

Tested with vanilla sage-4.4.3, Ubuntu 10.04, kernel 2.6.32-22-generic # 36-Ubuntu SMP Thu Jun 3 19:31:57 UTC 2010 x86_64 GNU/Linux, Intel(R) Core(TM) i5 CPU 650 @ 3.20GHz.

sage -t  "devel/sage-main/sage/rings/complex_mpc.pyx"       
     [1.8 s]

----------------------------------------------------------------------
All tests passed!

The script 'spkg-install' is a mix between the template from:

http://www.sagemath.org/doc/developer/producing_spkgs.html#creating-a-new-spkg

and the script from the mpfr spkg. In particular, I don't know what is the point of the EXTRA flags, but thought they might be useful.

You will also see in the file SPKG.txt:

== SPKG Maintainers ==

 * ???

I just can't be the maintainer, sorry...

bac7d3ea-3f1b-4826-8464-f0b53d5e12d2 commented 14 years ago

comment:35

It builds, OK on both

Solaris 10 03/2005 32-bit mode.
OpenSolaris 06/2009 in 64-bit mode.

But it fails tests on both machines.

Test failure on Solaris 10 03/2005

Configuration is

Sun Blade 1000
2 x 900 MHz UltraSPARC III+ CPUs
2 GB RAM
Solaris 10 03/05 (This was the first release of Solaris 10)
Sage 4.4.4.alpha1
mpc-0.8.2svn793.spkg (md5 checksum 2060cdf173efa2943529745438b26f1e)
32-bit build.

libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../src -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -O2 -MT read_data.lo -MD -MP -MF .deps/read_data.Tpo -c read_data.c  -fPIC -DPIC -o .libs/read_data.o
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../src -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -O2 -MT read_data.lo -MD -MP -MF .deps/read_data.Tpo -c read_data.c -o read_data.o >/dev/null 2>&1
mv -f .deps/read_data.Tpo .deps/read_data.Plo
/bin/bash ../libtool --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..  -I../src -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include   -O2 -MT comparisons.lo -MD -MP -MF .deps/comparisons.Tpo -c -o comparisons.lo comparisons.c
comparisons.o: No such file or directory
.libs/comparisons.o: No such file or directory
comparisons.lo: No such file or directory
comparisons.loT: No such file or directory
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../src -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -O2 -MT comparisons.lo -MD -MP -MF .deps/comparisons.Tpo -c comparisons.c  -fPIC -DPIC -o .libs/comparisons.o
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../src -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -O2 -MT comparisons.lo -MD -MP -MF .deps/comparisons.Tpo -c comparisons.c -o comparisons.o >/dev/null 2>&1
mv -f .deps/comparisons.Tpo .deps/comparisons.Plo
/bin/bash ../libtool --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..  -I../src -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include   -O2 -MT memory.lo -MD -MP -MF .deps/memory.Tpo -c -o memory.lo memory.c
memory.o: No such file or directory
.libs/memory.o: No such file or directory
memory.lo: No such file or directory
memory.loT: No such file or directory
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../src -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -O2 -MT memory.lo -MD -MP -MF .deps/memory.Tpo -c memory.c  -fPIC -DPIC -o .libs/memory.o
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../src -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -O2 -MT memory.lo -MD -MP -MF .deps/memory.Tpo -c memory.c -o memory.o >/dev/null 2>&1
mv -f .deps/memory.Tpo .deps/memory.Plo
/bin/bash ../libtool --tag=CC   --mode=link gcc  -O2  -L/export/home/drkirkby/sage-4.4.4.alpha1/local/lib -L/export/home/drkirkby/sage-4.4.4.alpha1/local/lib  -o libmpc-tests.la  random.lo tgeneric.lo read_data.lo comparisons.lo memory.lo  -lmpfr -lgmp 
libtool: link: ar cru .libs/libmpc-tests.a .libs/random.o .libs/tgeneric.o .libs/read_data.o .libs/comparisons.o .libs/memory.o 
libtool: link: ranlib .libs/libmpc-tests.a
libmpc-tests.la: No such file or directory
libtool: link: ( cd ".libs" && rm "libmpc-tests.la" && ln -s "../libmpc-tests.la" "libmpc-tests.la" )
libmpc-tests.la: No such file or directory
make[2]: *** [libmpc-tests.la] Error 2
make[2]: Leaving directory `/export/home/drkirkby/sage-4.4.4.alpha1/spkg/build/mpc-0.8.2svn793/src/tests'
make[1]: *** [check-am] Error 2
make[1]: Leaving directory `/export/home/drkirkby/sage-4.4.4.alpha1/spkg/build/mpc-0.8.2svn793/src/tests'
make: *** [check-recursive] Error 1
There was a problem during the mpc tests.
*************************************
Error testing package ** mpc-0.8.2svn793 **
*************************************
sage: An error occurred while testing mpc-0.8.2svn793

Test Failure on OpenSolaris 06/2009

Configuration is

Sun Ultra 27
Intel Xeon W3580 3.333 GHz, quad core, hyperthreaded (8 threads)
12 GB RAM
OpenSolaris 06/2009 (This is the latest release of OpenSolaris, but has updated to snv_134).
Sage 4.4.4.alpha1
mpc-0.8.2svn793.spkg (md5 checksum 2060cdf173efa2943529745438b26f1e)
64-bit build (SAGE64 was exported to "yes").

libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../src -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -O2 -g -m64 -O2 -g -m64 -O2 -MT comparisons.lo -MD -MP -MF .deps/comparisons.Tpo -c comparisons.c  -fPIC -DPIC -o .libs/comparisons.o
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../src -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -O2 -g -m64 -O2 -g -m64 -O2 -MT comparisons.lo -MD -MP -MF .deps/comparisons.Tpo -c comparisons.c -o comparisons.o >/dev/null 2>&1
mv -f .deps/comparisons.Tpo .deps/comparisons.Plo
/bin/sh ../libtool --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..  -I../src -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -O2 -g -m64   -O2 -g -m64   -O2 -MT memory.lo -MD -MP -MF .deps/memory.Tpo -c -o memory.lo memory.c
rm: memory.o: No such file or directory
rm: .libs/memory.o: No such file or directory
rm: memory.lo: No such file or directory
rm: memory.loT: No such file or directory
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../src -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -O2 -g -m64 -O2 -g -m64 -O2 -MT memory.lo -MD -MP -MF .deps/memory.Tpo -c memory.c  -fPIC -DPIC -o .libs/memory.o
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../src -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -I/export/home/drkirkby/sage-4.4.4.alpha1/local/include -O2 -g -m64 -O2 -g -m64 -O2 -MT memory.lo -MD -MP -MF .deps/memory.Tpo -c memory.c -o memory.o >/dev/null 2>&1
mv -f .deps/memory.Tpo .deps/memory.Plo
/bin/sh ../libtool --tag=CC   --mode=link gcc  -O2 -g -m64   -O2  -L/export/home/drkirkby/sage-4.4.4.alpha1/local/lib -L/export/home/drkirkby/sage-4.4.4.alpha1/local/lib  -o libmpc-tests.la  random.lo tgeneric.lo read_data.lo comparisons.lo memory.lo  -lmpfr -lgmp 
libtool: link: ar cru .libs/libmpc-tests.a .libs/random.o .libs/tgeneric.o .libs/read_data.o .libs/comparisons.o .libs/memory.o 
libtool: link: ranlib .libs/libmpc-tests.a
rm: libmpc-tests.la: No such file or directory
libtool: link: ( cd ".libs" && rm "libmpc-tests.la" && ln -s "../libmpc-tests.la" "libmpc-tests.la" )
rm: libmpc-tests.la: No such file or directory
make[2]: *** [libmpc-tests.la] Error 2
make[2]: Leaving directory `/export/home/drkirkby/sage-4.4.4.alpha1/spkg/build/mpc-0.8.2svn793/src/tests'
make[1]: *** [check-am] Error 2
make[1]: Leaving directory `/export/home/drkirkby/sage-4.4.4.alpha1/spkg/build/mpc-0.8.2svn793/src/tests'
make: *** [check-recursive] Error 1
There was a problem during the mpc tests.
*************************************
Error testing package ** mpc-0.8.2svn793 **
*************************************
sage: An error occurred while testing mpc-0.8.2svn793
Please email sage-devel http://groups.google.com/group/sage-devel
explaining the problem and send the relevant part of
of /export/home/drkirkby/sage-4.4.4.alpha1/install.log.  Describe your computer, operating system, etc.
If you want to try to fix the problem yourself, *don't* just cd to
/export/home/drkirkby/sage-4.4.4.alpha1/spkg/build/mpc-0.8.2svn793 and type 'make check' or whatever is appropriate.
Instead, the following commands setup all environment variables
correctly and load a subshell for you to debug the error:
(cd '/export/home/drkirkby/sage-4.4.4.alpha1/spkg/build/mpc-0.8.2svn793' && '/export/home/drkirkby/sage-4.4.4.alpha1/sage' -sh)
When you are done debugging, you can type "exit" to leave the
subshell.

real    0m12.719s
user    0m8.046s
sys 0m3.403s

2c00b582-cfe9-43b9-8124-fec470060704 commented 14 years ago

comment:36

ok, my fault, I didn't tried the spkg-check. (is the one in the mpfr spkg working???) It should be corrected with the brand new

http://yann.laiglechapuy.net/spkg/mpc-0.8.3-dev-svn793.spkg

at least, it works for me... same proc, os, etc

===================
All 57 tests passed
===================

bac7d3ea-3f1b-4826-8464-f0b53d5e12d2 commented 14 years ago

Reviewer: William Stein, David Kirkby

bac7d3ea-3f1b-4826-8464-f0b53d5e12d2 commented 14 years ago

comment:37

Replying to @sagetrac-ylchapuy:

ok, my fault, I didn't tried the spkg-check. (is the one in the mpfr spkg working???) It should be corrected with the brand new

http://yann.laiglechapuy.net/spkg/mpc-0.8.3-dev-svn793.spkg

at least, it works for me... same proc, os, etc
===================
All 57 tests passed
===================

mpc-0.8.3-dev-svn793.spkg (md5 checksum 07e1b56fe1e551138b8862d5551d6948) passed all 57 tests on both machines I have tested on.

Sun Blade 1000, UltraSPARC III+ processors, Solaris 10 03/2005 32-bit mode.
Sun Ultra 27, quad core Xeon processor, OpenSolaris 06/2009 in 64-bit mode.

However, it seems a long way before this could get a positive review and be incorporated into Sage.

William has made quite extensive comments - in particular the fact it could not be "optional" so would have to be a standard part of Sage.
Nobody has agreed to maintain it. I believe there is a requirement for any standard package that someone agrees to maintain it for 2 years.
For the package to become "standard" it needs a vote.
I don't like the idea of using a SVN snapshot unless necessary. This is not based on a stable release of MPC, yet I believe it would have to become a standard part of Sage. That seems a bad idea in my opinion.
It appears to have undergone very little testing. When I noticed the test suite had failed, you remarked you had not run the tests. When several months ago I asked if it had been tested on Solaris, nobody responded.
Has it been tested on OS X? If so, what processors?

Are you aware Sage is suppose to support all these platforms. What subset of these has MPC been tested on?

PROCESSOR        OPERATING SYSTEM
x86              32-bit Linux -- Debian, Ubuntu, CentOS (=Red Hat),
                               Fedora, openSUSE, Mandriva, Arch
x86_64           64-bit Linux -- Debian, Ubuntu, CentOS (=Red Hat),
                               Fedora, openSUSE, Mandriva, Arch
IA-64 Itanium 2  64-bit Linux -- Red Hat, SUSE
x86              Apple Mac OS X 10.4.x, 10.5.x, 10.6.x
PPC              Apple Mac OS X 10.4.x, 10.5.x, 10.6.x
SPARC            Solaris 10

Have the doc tests been run on different hardware? Would results be affected by different floating point processors, so that the doc tests might need to consider this.

I would add, I have a lot of respect for Paul Zimmermann, so I would tend to trust code he is associated with.

It just strikes me that this is going to need a lot more than one positive review to become part of Sage. I believe any competent reviewer will ask for more proof this would be a good addition to Sage. At the moment, whilst the fact Paul is associated with this code would tend to make me think it is well tested, as a package in Sage, there seems to be little evidence presented that this has undergone much testing. Given its based on a snapshot, makes me even more suspicious.

A few others points.

One of the attachments says it is released under the GPL, but does not state the version. I believe it should say "GPL version 2, or (at your option) any later version."
As for your question about the MPFR test suite, if you look at MPFR's spkg-install you will see that the test suite is run every time Sage is built. That is because the test suite tended to fail quite regularly with buggy compilers, or buggy operating systems, so it was wise to test it on every Sage build.
What does the '793' in the .spkg mean? I assumed at first it was a revision, but now see that is unchanged, whilst the number in the package has changed from 0.8.2 to 0.8.3.

Dave

zimmermann6 commented 14 years ago

comment:38

thank you David for your comments on this ticket. We (the MPC developers) have invested a lot of time on this ticket (especially Philippe Theveny in 2008-2009, where at that time he was an engineer paid by INRIA to work on MPFR and MPC), thus we would appreciate a lot if the Sage developers could tell us right now in case they believe MPC will never become a standard package, so that we don't spend more time on this ticket.

As for the technical issues, I can answer a few ones:

this package is based on svn version 793 just to speed up the mpc_pow_si computations. Note that if the Sage developers had requested this before, this could have been done for the last release (0.8.2). The changes wrt the 0.8.2 release are therefore minimal.
about testing, you surely know that MPC is now a prerequisite to compile GCC 4.5 (see http://gcc.gnu.org/install/prerequisites.html). You will find an extensive list of supported platforms on http://www.multiprecision.org/index.php?prog=mpc&page=platforms.
about doctests, MPC is based on MPFR, which produces platform-independent results, thus the doctests do not depend on the platform, contrary to other numeric packages. This is one of the reasons MPC was chosen by GCC for constant folding.
MPC is released under LGPL v2.1+.

Paul Zimmermann

2c00b582-cfe9-43b9-8124-fec470060704 commented 14 years ago

comment:39

Hi David, Paul already gave you some answers, I will complete with my own point of view:

first just to be clear, I am not related in any way with the MPC team. I started updating this spkg from the one made by Philippe Theveny because I do think it would be a good thing for the Sage project to have it;
regarding the "optional" problem; I just extended what Philippe does. I personally don't know how to make an optional package with Cython interfaces to a C library, if you know how to do it please give me some pointers;
I know it's the policy now to have someone agreeing to maintain spkg's for some time; but I just can't and live this to others;
as explained by Paul, the svn revision 793 respond to a request from Sage developers. The problem with the bad version numbering (0.8.2 instead of 0.8.3-dev) comes from me, sorry for this;
I have no access to other platforms, and don't have more time to spend (I know I could ask William for an account for Sage development);
the part relying purely on MPC should be platform oblivious; of course the part interacting with Sage CC stuff can still have some rounding problems. None of this has been tested on different platforms (by me at least);
finally, regarding the license, the file SPKG.txt is clear with this. I didn't check the patches attached with this ticket.

As a conclusion, I want to say that I won't have much more time to spend on this. If there is a clear way not to transform those 20 months in a big waste of time I would be please to make a last effort.

Yann

PS: the SPKG changed once again. The problem with the previous spkg-check was just the usual RM and libtool glitch. My corrected script was doing a lot of work for nothing. The md5 sum is now af419b37f887082a2979ce3a9d0c194a .

sagemath / sage

New module complex_mpc using lib mpc for complex multiprecision arithmetic #4446

Test failure on Solaris 10 03/2005

Test Failure on OpenSolaris 06/2009