Closed b12b007b-ced8-4aef-b4e9-5b123b0bb8b3 closed 14 years ago
Attachment: lcalc-1.23.p1.spkg.gz
For lcalc wrapper. This should work with the old code based on command line version of lcalc
I am attaching another spkg. This is based on the earlier lcalc spkg. This should work. This also follows the instructions of http://www.sagemath.org/doc/developer/producing_spkgs.html
Special thanks to John from whom I have learnt a lot during this review process.
I realized that instead of having Makefile.sage, having just a patch file will make it more robust to changes in Makefile by Mike later.
Changed reviewer from John Cremona to John Cremona, David Kirkby
Attachment: lcalc-1.23.p2.spkg.gz
William said the other day he does not want to see patches used. One reason cited was the fact that not all systems have a patch command, though I pointed out 'patch' was a requirement for POSIX.
As such, I think you should rename this to a makefile.patch, and go back to your original solution of a patched makefile in the patches directory.
Following David Kirby's comment, I have removed the changes from p2. This is exactly the same as lcalc-1.23.p1
Attachment: lcalc-1.23.p3.spkg.gz
I have removed the patch file, and gone back to the version 1.23.p1. This works on boxen. All the tests passed.
I'm a bit concerned about the need to change the doctest to check for a result which is very different from the original, so the doctest chances from
0.305999824716...
to
0.305999..
I know numerical noise does occur in calculations, but this seems a lot of it. Is there independant verification the new result is correct (in which case let's use it, and not all the dots), or is there independant verification the old result is correct, in which case it is doubtful this should be put in.
I've expressed concern before about the way lcalc hides compiler warnings, which happened to be in a way that it stopped the Solaris build #6609. It basically added a compiler flag, which was supposed to be sent to the gnu assembler, to suppress warnings.
There are also #7178 and #7065 which I've raised.
The fact this is so such a different result to the previous one, deserves some explanation I feel. I'd like to see an independent check with Mathematica.
I'm marking it as 'needs info', as I think we should not just ignore the different result, but understand why it is so different.
Dave
I'm not a mathematician, but do have access to Mathematica. Can anyone tell me what this means, so I can compute this value in Mathematica? If I increase the precision, I should be able to get an accurate result.
I have a comment regarding the remark by John Cremona about a different numerical value, and whether this could be due to his use of a 32-bit computer. This looks all like floating point maths to me, so the result would depend only on the IEEE floating point standard, which should give identical results on any Intel or AMD CPU,. as they both use 64-bits for the result, but work at 80 bits internally. This makes me even more concerned, and believe this needs addressing.
Dave
I don't think Mathematica will be much help here: evaluating derivatives of L-series of elliptic curves.
I tried Magma, but as far as I can see they don't do derivatives. They can evaluate the L-series, and I did a "poor man's derivative" evaluating (L(0.5+e)-L(0.5))/e for small e and the result seemed to converge to 0.017188297766 and not anything like 0.305.
So Dave's question about what's going on here does need to be answered.
0.305... looks more like the value of L.derivative(1).
Replying to @mwhansen:
0.305... looks more like the value of L.derivative(1).
You're right! That's the value I have in my database, anyway. It is possible that lcalc normalises the functions so that the centre of the critical strip is 0.5 and not 1.
So I can verify with my own program that the value to around 50 decimals is
0.305999773834052301820483683321676474452637774590
And in Magma (once I read the manual):
> E := EllipticCurve([0,0,1,-1,0]);
> CremonaReference(E);
37a1
> L := LSeries(E);
> Evaluate(L,1 : Derivative:=1);
0.305999773834052301820483683322
As you can tell, this is outside my Mathematical knowledge, but just a few comments.
Dave
Yes it does normalize so that the center is at 0.5. The derivatives are not precise. I have added Mike to the discussion list so that he can give more details.
I should have mentioned it in the last posting. I was planning to create another layer of python lfuntion class which will have the standard center (for general newforms).
Replying to @sagetrac-drkirkby:
As you can tell, this is outside my Mathematical knowledge, but just a few comments.
- If this behaves differently to what someone might expect, then it should be documented how it works. I believe John is an expert in this area and he was surprised by the behavior, then clearly it could cause confusion.
The center being at half is documented. The key line is \Lambda(s) = \omega Q^s \overline{\Lambda(1-\bar s)}
- Since John's computed an computed a result with high precision, I would put that as a comment in the test code.
lcalc uses double to compute. The high precision is not possible. There is another ticket which contains documentation about loss of precision when calculating derivatives.
- I recall William once saying lcalc was very fast, but to me at least, this is not computing a very accurate result. I would suggest that should also be documented too. along with slower, but more accurate alternatives.
Using double makes things fast, but they will not be as accurate as using a multiprecision library.
Dave
I think it is important to note here that lcalc can handle a vastly larger assortment of L-functions than the opposition. For example, the one value I recomputed independently is of the only kind I can handle with my code (and I can only compute the value at the centre, not anywhere else!). So we should not leave the impression that lcalc is inferior just because it only uses double precision and not multi.
Replying to @JohnCremona:
I think it is important to note here that lcalc can handle a vastly larger assortment of L-functions than the opposition. For example, the one value I recomputed independently is of the only kind I can handle with my code (and I can only compute the value at the centre, not anywhere else!). So we should not leave the impression that lcalc is inferior just because it only uses double precision and not multi.
I agree, but there is significant precision loss - perhaps more than some might expect, so personally I would have thought that worth documenting. Of course, anyone should know hardware floating point will be less accurate (but faster) than extended precision in software.
Given for simple computations, 16 digits of precision are possible, but here only 9, I would have thought that work documenting.
The fact lcalc can handle a vastly larger assortment of L-functions than anything else should also be noted!
Dave
Replying to @sagetrac-drkirkby:
Replying to @JohnCremona:
I think it is important to note here that lcalc can handle a vastly larger assortment of L-functions than the opposition. For example, the one value I recomputed independently is of the only kind I can handle with my code (and I can only compute the value at the centre, not anywhere else!). So we should not leave the impression that lcalc is inferior just because it only uses double precision and not multi.
I agree, but there is significant precision loss - perhaps more than some might expect, so personally I would have thought that worth documenting. Of course, anyone should know hardware floating point will be less accurate (but faster) than extended precision in software.
Given for simple computations, 16 digits of precision are possible, but here only 9, I would have thought that work documenting.
The fact lcalc can handle a vastly larger assortment of L-functions than anything else should also be noted!
Dave
Description changed:
---
+++
@@ -1 +1,77 @@
-Wrapping lcalc library
+
+I plan to release version L-1.3 in a couple of weeks, and also to
+have Rishi update his cython wrapper during our march workshop (I made
+a handful of changes that will need him to update).
+
+There are a number of key improvements.
+
+Precision-
+
+I've spent quite a bit of effort getting a multiprecision version
+of lcalc to work. I now have it working with Bailey's double double
+and quad double (about 30 and 60 digits respectively) and also MPFR,
+though the latter seems quite slow, perhaps because of the c++ wrapper that I am using... haven't dug too deep with the latter since
+Bailey's dd and qd works quite nicely, and 60 digits is quite reasonable. Nonetheless,
+it does work to higher precision with mpfr.
+
+I am also developing a test suite to be run at compile time to check on accuracy of many of the routines and do various timing tests.
+
+Output precision-
+
+Rishi needs to handle this carefully since I have a parameter
+which describes how many digits of the working digits are accurate.
+I use this parameter when outputting my results. Not sure how
+or if Rishi makes use of this extra info, or just passes the double results
+of my routines.
+
+At the same time I am wiring in better control of output precision
+based on the explicit formula and also on using the smooth approximate
+functional equation with a couple of different test functions.
+
+Derivatives-
+
+the version in L-1.23 is very crummy and I fixed that up too
+in version L-1.3. I initially just wanted to get that functionality in there, and I now use (version L-1.3 to be released) higher central differences for the derivatives that gives an accuracy of about Digits*4/(n+4) for the nth derivative. So for the 1st derivative,
+my new version is accurate to about 4/5 the working precision, so roughly 12-13 digits with double precision and 24-25 digits with Bailey's dd.
+
+
+Zeros-
+
+I fixed a subtle bug that would on rare occasion cause some zeros to be omitted
+and other zeros to be duplicated in their place.
+
+I'm also finishing to wire in a test of the explicit formula (based on a prototype that Kevin McGown helped me with in our kickoff frg coding sprint) that will prevent
+such bugs from going unnoticed!
+
+Trivial zeros-
+
+I built in knowledge of the trivial zeros so that correct output is displayed there.
+This allowed me to apply the functional equation correctly, so that the new version
+works throughout the complex plane (previously it gave nonsense results left of 0).
+
+Band limited interpolation-
+
+I'm wiring in some code of ghaith Hiary for computing zeta using band limited
+interpolation. This will give a significant speedup in, for example, computing zeros of zeta.
+
+Start at Nth zero-
+
+Ability to start searching for zeros from the Nth zero onward (rather than
+always starting from the real axis). This one was long overdue!
+
+
+Many small improvements that have a big impact-
+
+For example I coded up my own trig functions that are about 4 times faster than
+the machines (for double precision) and also faster than Bailey's and MPFR's.
+
+Anyhow, my package definitely needs a companion paper and much better
+documentation. Something to work on...
+
+Other planned improvements for version L-1.4:
+
+1) broader use of openmp in some of the key routines
+2) fft algorithms- will lead to orders of magnitude improvement
+especially for higher degree L-funcitons
+3) L-functions of number fields and symmetric powers
+
I plan to release version L-1.3 in a couple of weeks, and also to have Rishi update his cython wrapper during our march workshop (I made a handful of changes that will need him to update).
There are a number of key improvements.
Precision-
I've spent quite a bit of effort getting a multiprecision version of lcalc to work. I now have it working with Bailey's double double and quad double (about 30 and 60 digits respectively) and also MPFR, though the latter seems quite slow, perhaps because of the c++ wrapper that I am using... haven't dug too deep with the latter since Bailey's dd and qd works quite nicely, and 60 digits is quite reasonable. Nonetheless, it does work to higher precision with mpfr.
I am also developing a test suite to be run at compile time to check on accuracy of many of the routines and do various timing tests.
Output precision-
Rishi needs to handle this carefully since I have a parameter which describes how many digits of the working digits are accurate. I use this parameter when outputting my results. Not sure how or if Rishi makes use of this extra info, or just passes the double results of my routines.
At the same time I am wiring in better control of output precision based on the explicit formula and also on using the smooth approximate functional equation with a couple of different test functions.
Derivatives-
the version in L-1.23 is very crummy and I fixed that up too in version L-1.3. I initially just wanted to get that functionality in there, and I now use (version L-1.3 to be released) higher central differences for the derivatives that gives an accuracy of about Digits*4/(n+4) for the nth derivative. So for the 1st derivative, my new version is accurate to about 4/5 the working precision, so roughly 12-13 digits with double precision and 24-25 digits with Bailey's dd.
Zeros-
I fixed a subtle bug that would on rare occasion cause some zeros to be omitted and other zeros to be duplicated in their place.
I'm also finishing to wire in a test of the explicit formula (based on a prototype that Kevin McGown helped me with in our kickoff frg coding sprint) that will prevent such bugs from going unnoticed!
Trivial zeros-
I built in knowledge of the trivial zeros so that correct output is displayed there. This allowed me to apply the functional equation correctly, so that the new version works throughout the complex plane (previously it gave nonsense results left of 0).
Band limited interpolation-
I'm wiring in some code of ghaith Hiary for computing zeta using band limited interpolation. This will give a significant speedup in, for example, computing zeros of zeta.
Start at Nth zero-
Ability to start searching for zeros from the Nth zero onward (rather than always starting from the real axis). This one was long overdue!
Many small improvements that have a big impact-
For example I coded up my own trig functions that are about 4 times faster than the machines (for double precision) and also faster than Bailey's and MPFR's.
Anyhow, my package definitely needs a companion paper and much better documentation. Something to work on...
Other planned improvements for version L-1.4:
1) broader use of openmp in some of the key routines 2) fft algorithms- will lead to orders of magnitude improvement especially for higher degree L-funcitons 3) L-functions of number fields and symmetric powers
Description changed:
---
+++
@@ -1,77 +1 @@
-
-I plan to release version L-1.3 in a couple of weeks, and also to
-have Rishi update his cython wrapper during our march workshop (I made
-a handful of changes that will need him to update).
-
-There are a number of key improvements.
-
-Precision-
-
-I've spent quite a bit of effort getting a multiprecision version
-of lcalc to work. I now have it working with Bailey's double double
-and quad double (about 30 and 60 digits respectively) and also MPFR,
-though the latter seems quite slow, perhaps because of the c++ wrapper that I am using... haven't dug too deep with the latter since
-Bailey's dd and qd works quite nicely, and 60 digits is quite reasonable. Nonetheless,
-it does work to higher precision with mpfr.
-
-I am also developing a test suite to be run at compile time to check on accuracy of many of the routines and do various timing tests.
-
-Output precision-
-
-Rishi needs to handle this carefully since I have a parameter
-which describes how many digits of the working digits are accurate.
-I use this parameter when outputting my results. Not sure how
-or if Rishi makes use of this extra info, or just passes the double results
-of my routines.
-
-At the same time I am wiring in better control of output precision
-based on the explicit formula and also on using the smooth approximate
-functional equation with a couple of different test functions.
-
-Derivatives-
-
-the version in L-1.23 is very crummy and I fixed that up too
-in version L-1.3. I initially just wanted to get that functionality in there, and I now use (version L-1.3 to be released) higher central differences for the derivatives that gives an accuracy of about Digits*4/(n+4) for the nth derivative. So for the 1st derivative,
-my new version is accurate to about 4/5 the working precision, so roughly 12-13 digits with double precision and 24-25 digits with Bailey's dd.
-
-
-Zeros-
-
-I fixed a subtle bug that would on rare occasion cause some zeros to be omitted
-and other zeros to be duplicated in their place.
-
-I'm also finishing to wire in a test of the explicit formula (based on a prototype that Kevin McGown helped me with in our kickoff frg coding sprint) that will prevent
-such bugs from going unnoticed!
-
-Trivial zeros-
-
-I built in knowledge of the trivial zeros so that correct output is displayed there.
-This allowed me to apply the functional equation correctly, so that the new version
-works throughout the complex plane (previously it gave nonsense results left of 0).
-
-Band limited interpolation-
-
-I'm wiring in some code of ghaith Hiary for computing zeta using band limited
-interpolation. This will give a significant speedup in, for example, computing zeros of zeta.
-
-Start at Nth zero-
-
-Ability to start searching for zeros from the Nth zero onward (rather than
-always starting from the real axis). This one was long overdue!
-
-
-Many small improvements that have a big impact-
-
-For example I coded up my own trig functions that are about 4 times faster than
-the machines (for double precision) and also faster than Bailey's and MPFR's.
-
-Anyhow, my package definitely needs a companion paper and much better
-documentation. Something to work on...
-
-Other planned improvements for version L-1.4:
-
-1) broader use of openmp in some of the key routines
-2) fft algorithms- will lead to orders of magnitude improvement
-especially for higher degree L-funcitons
-3) L-functions of number fields and symmetric powers
-
+Wrapping lcalc library
One more thing- I've also cleaned up my makefile a bit, and, in the source, got rid of depreciated headers and unused variables.
Replying to @sagetrac-mrubinst:
One more thing- I've also cleaned up my makefile a bit, and, in the source, got rid of depreciated headers and unused variables.
That is really excellent news. Is there any chance either you or I could check this with the Sun compiler on Solaris, before you make an official release? The Sun compilers more fussy than gcc, but would give us a performance boost on Solaris.
The fact the compiler is more fussy, also tends to weed out some sort of bugs. On 't2' the Sun C compiler is
/opt/sunstudio12.1/bin/cc
and the C++ compiler is
/opt/sunstudio12.1/bin/CC
On the C++ compiler, you will probably need the option
-library=stlport4
as that will then implement the latest C++ standard. (The older C++ standard is not compatible with the latest standard. For backwards compatibility Sun's C++ compiler defaults to the old C++ standard.}}}
Previously I tried building lcalc with the Sun compiler but got lots of warnings and error messages. But if the code has been cleaned up, things have more chance of working.
The point I made above about storing the exact (or least high precision) result in the test file, would be that is some record of what and should be, even though it is not going to be achieved.
Then if you bring out a later version of lcalc, which deviates more from the exact result, people will at least know. It would only add 70 or so bytes to a file.
Dave
Has this been checked on Solaris 10 with gcc? The different floating point processor may well give a different answer and need revisions to the doc test.
There's general information about building on Solaris at
http://wiki.sagemath.org/solaris
Information specifically for 't2' at
http://wiki.sagemath.org/devel/Building-Sage-on-the-T5240-t2
Both the source (4.3.0.1 is the latest to build on Solaris) and a binary which will run on any SPARC can be found at http://www.sagemath.org/download-source.html
Dave
I am unable to decipher the following error message when I try to compile on t2.
g++ -Wa,-W -O3 -Wno-deprecated -ffast-math -fPIC -I../include Lglobals.o Lgamma.o Lriemannsiegel.o Lriemannsiegel_blfi.o Ldokchitser.o Lcommandline_globals.o Lcommandline_misc.o Lcommandline_numbertheory.o Lcommandline_values_zeros.o Lcommandline_elliptic.o Lcommandline_twist.o Lcommandline.o cmdline.o -Xlinker -export-dynamic -Xlinker -rpath -Xlinker /scratch/rishi/sage-4.3.0.1-Solaris-10-SPARC-sun4u-or-sun4v/local/lib -L/scratch/rishi/sage-4.3.0.1-Solaris-10-SPARC-sun4u-or-sun4v/local/lib -lpari -o lcalc
ld: fatal: option -dn and -P are incompatible
ld: fatal: Flags processing errors
collect2: ld returned 1 exit status
make[1]: *** [lcalc] Error 1
make[1]: Leaving directory `/scratch/rishi/sage-4.3.0.1-Solaris-10-SPARC-sun4u-or-sun4v/spkg/build/lcalc-1.23.p3/src/src'
make: *** [all] Error 2
I have not seen -dn or -P options in anything.
Attachment: lcalc-1.23.p4.spkg.gz
Changed makefile to compile on solaris
I have attached new spkg. This compiles on solaris and the tests passed on t2.
Replying to @sagetrac-drkirkby:
Replying to @sagetrac-mrubinst:
One more thing- I've also cleaned up my makefile a bit, and, in the source, got rid of depreciated headers and unused variables.
That is really excellent news. Is there any chance either you or I could check this with the Sun compiler on Solaris, before you make an official release? The Sun compilers more fussy than gcc, but would give us a performance boost on Solaris.
Having looked at this more closely, you have suppressed warnings, after I'd gone out of the way to enable them.
You have the options -W and -Wno-deprecated, which are documented at
http://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#Warning-Options
as
-w
Inhibit all warning messages.
-Wno-deprecated
Do not warn about usage of deprecated features. See Deprecated Features.
If we follow that link about Deprecated Features, it says "Using the old features might cause a warning in some cases that the feature will be dropped in the future. In other cases, the feature might be gone already."
So this appears to be storing up trouble for the future to me.
Given I'd noticed some problems before on Solaris, I made an effort to highlight the warnings. Now they have gone, as you have suppressed them.
As more minor points:
Dave
Dave
It would be good if you could at least add -Wall and get rid of the warnings, and optionally -pedantic too. The the code will hopefully not break any time soon. This could easily break with a compiler upgrade.
Dave
This version is not what Mike was writing about in his posting. He was giving a preview of a future release of lcalc. This version is not as polished. But then it is better than what is shipping with sage right now.
Regarding overwriting older packages, John Cremona asked that I do exactly the opposite of what you are suggesting. Please see his posting from couple of months ago in this very ticket. I will include a one line description of p3. p3 was just reverting from p2 to p1.
I just made the minimal changes to Mike's makefile so that it compiles on solaris and uses pari library in SAGE_LOCAL. I did not want to make too many changes to the lcalc makefile.
In any case I will make changes you suggested to Makefile.sage in patches directory and post it.
Replying to @rishikesha:
This version is not what Mike was writing about in his posting. He was giving a preview of a future release of lcalc. This version is not as polished. But then it is better than what is shipping with sage right now.
I should clarify that I am talking about lcalc the program, and not the build scripts.
Replying to @rishikesha:
Regarding overwriting older packages, John Cremona asked that I do exactly the opposite of what you are suggesting. Please see his posting from couple of months ago in this very ticket. I will include a one line description of p3. p3 was just reverting from p2 to p1.
I'm really confused here. I'm going to cc Minh onto this, as he knows more about version numbers than me, but I believe the version being proposed is incorrect.
6 months ago you added 'L-1.23.spkg' to this ticket. So was that supposed to be brand new package? Currently there is no package in Sage called 'L'. In any case it should not have been attacked to the trac server, but instead a link provided to a place where it could be found.
There are numerous other attachments. Assuming all these changes are based on top of version 1.23 of lcalc, and not a version with a date code of'20080205', then I believe all these changes should be put into lcalc-1.23. There is no need for p0, p1, p2, p3 or p4. Patch levels should be incremented as a result of newer versions committed to Sage - not as a result of reviewer comments.
If all these changes are based on the version of lcalc in Sage, lcalc-20080205.p4.spkg, then the updated version should be called lcalc-20080205.p5.spkg.
Perhaps you could briefly summerise what this is, and why it should be called a particular version. Why was it orriginally going to be called L? With a highly technical package and 70 comments spanning 6-months, it is hard to decipher it all. But I believe something is wrong here, but would welcome some comments from Minh.
Dave
Replying to @sagetrac-drkirkby:
Replying to @rishikesha:
Regarding overwriting older packages, John Cremona asked that I do exactly the opposite of what you are suggesting. Please see his posting from couple of months ago in this very ticket. I will include a one line description of p3. p3 was just reverting from p2 to p1.
I'm really confused here. I'm going to cc Minh onto this, as he knows more about version numbers than me, but I believe the version being proposed is incorrect. * 6 months ago you added 'L-1.23.spkg' to this ticket. So was that supposed to be brand new package? Currently there is no package in Sage called 'L'. In any case it should not have been attacked to the trac server, but instead a link provided to a place where it could be found.
Please read the comments in this ticket. #5396 comment:39
There are numerous other attachments. Assuming all these changes are based on top of version 1.23 of lcalc, and not a version with a date code of'20080205', then I believe all these changes should be put into lcalc-1.23. There is no need for p0, p1, p2, p3 or p4. Patch levels should be incremented as a result of newer versions committed to Sage - not as a result of reviewer comments.
If all these changes are based on the version of lcalc in Sage, lcalc-20080205.p4.spkg, then the updated version should be called lcalc-20080205.p5.spkg.
It is not based on lcalc-20080205.
Perhaps you could briefly summerise what this is, and why it should be called a particular version. Why was it orriginally going to be called L? With a highly technical package and 70 comments spanning 6-months, it is hard to decipher it all. But I believe something is wrong here, but would welcome some comments from Minh.
Dave
HOWTO
This patch no longer compiles with the latest version of sage.
(minor) rebase for sage4.3.3
Attachment: trac_5396-rebase_sage4.3.3.patch.gz
You just have to apply the one character patch trac_5396-rebase_sage4.3.3.patch on top of others.
combined patch rebased to 4.3.3
Attachment: trac_5396_combined_rebasedto_4.3.3.patch.gz
Oops! I did not see Yann had already posted a patch.
This ticket has become too long. I am splitting the lcalc spkg to a ticket originally by mabshoff #4793
Only the last patch trac_5396_combined_rebasedto_4.3.3.patch is needed. It combines all the work prior to it. This patch depends spkg upgrade in #4793
Attachment: trac_5396_combined_rebased_4.3.4.patch.gz
The location of header files have changed in the lcalc spkg. This incorporates neccessary changes to compile.
Description changed:
---
+++
@@ -1 +1 @@
-Wrapping lcalc library
+Wrapping lcalc library. This patch depends spkg upgrade in #4793
Since the spkg upgrade this patch depended on has received positive review, can someone reinstate the positive review here.
Applied and tested on sage-4.4.2.
Changed reviewer from John Cremona, David Kirkby to John Cremona, David Kirkby, Alex Ghitza
Changed author from rishi, ylchapuy to Rishikesh, Yann-Laigle Chapuy
Wrapping lcalc library. This patch depends spkg upgrade in #4793
CC: @craigcitro @JohnCremona @sagetrac-bober mrubinst@math.uwaterloo.ca @sagetrac-mvngu
Component: number theory
Keywords: lcalc
Author: Rishikesh, Yann Laigle-Chapuy
Reviewer: John Cremona, David Kirkby, Alex Ghitza
Merged: sage-4.5.2.alpha1
Issue created by migration from https://trac.sagemath.org/ticket/5396