Closed jdemeyer closed 12 years ago
Description changed:
---
+++
@@ -1 +1,3 @@
The aim is to add a gcc package to Sage. We probably don't want to always build it, we would install it by default only on systems where this is needed.
+
+**spkg**: [http://boxen.math.washington.edu/home/jdemeyer/spkg/gcc-4.4.6.spkg](http://boxen.math.washington.edu/home/jdemeyer/spkg/gcc-4.4.6.spkg)
Description changed:
---
+++
@@ -1,3 +1,3 @@
-The aim is to add a gcc package to Sage. We probably don't want to always build it, we would install it by default only on systems where this is needed.
+The aim is to add a gcc package to Sage. We probably don't want to always build it, we would install it by default only on systems where this is needed. This would replace the fortran package.
**spkg**: [http://boxen.math.washington.edu/home/jdemeyer/spkg/gcc-4.4.6.spkg](http://boxen.math.washington.edu/home/jdemeyer/spkg/gcc-4.4.6.spkg)
Changed dependencies from #12366, #12367, #12368 to #12366, #12367, #12368, #10492
Description changed:
---
+++
@@ -1,3 +1,5 @@
The aim is to add a gcc package to Sage. We probably don't want to always build it, we would install it by default only on systems where this is needed. This would replace the fortran package.
**spkg**: [http://boxen.math.washington.edu/home/jdemeyer/spkg/gcc-4.4.6.spkg](http://boxen.math.washington.edu/home/jdemeyer/spkg/gcc-4.4.6.spkg)
+
+**Apply** [attachment: 12369_gcc_root.patch](https://github.com/sagemath/sage-prod/files/10654635/12369_gcc_root.patch.gz) to the `SAGE_ROOT` repository.
Changed dependencies from #12366, #12367, #12368, #10492 to #11073, #12366, #12367, #12368, #10492, #12405
# Install gcc if the system-provided one is older than gcc-4.4.
But that is all Macs. This seems overkill to me; why not only do this on Lion?
Or would this eliminate having Xcode as a prerequisite to building Sage on OS X in general (even for those for which XCode is free, though sometimes annoying to get)? That might be a nice move. I assume this is a prebuilt gcc like our current Fortran package?
Could the message "installing GCC because..." occur several times here?
Updated spkg at http://www.stp.dias.ie/~vbraun/Sage/spkg/gcc-4.6.2.spkg
It turns out to be tricky to convince gcc to use lib64
for glibc but put its own libraries into lib
. But I think its safe to just move the generated shared libraries to SAGE_LOCAL/lib
. Also, I updated to the newest gcc, install in single-thread, and disabled multilib.
First of all: this spkg is all pretty-much proof-of-concept. I haven't even managed to install Sage from source with this spkg.
Replying to @kcrisman:
# Install gcc if the system-provided one is older than gcc-4.4.
But that is all Macs. This seems overkill to me; why not only do this on Lion?
Maybe you are right, but my idea was to install this gcc spkg if the gcc we ship is more recent than the one the user has. I think many people would benefit from using a more recent compiler.
Or would this eliminate having Xcode as a prerequisite to building Sage on OS X in general (even for those for which XCode is free, though sometimes annoying to get)? That might be a nice move. I assume this is a prebuilt gcc like our current Fortran package?
No, this is a gcc from source. I certainly don't want to include gcc binaries for every platform, that would increase the size way too much.
Could the message "installing GCC because..." occur several times here?
Sure.
First of all: this spkg is all pretty-much proof-of-concept. I haven't even managed to install Sage from source with this spkg.
Of course; I'm just asking early questions.
But that is all Macs. This seems overkill to me; why not only do this on Lion?
Maybe you are right, but my idea was to install this gcc spkg if the gcc we ship is more recent than the one the user has. I think many people would benefit from using a more recent compiler.
Possibly. I guess what I'm wondering is how one would build this gcc if one didn't have gcc... but this may betray my ignorance. Anyway, the vast majority of Mac users will not have a way to build this gcc if they need gcc until they download/install (from a disk which they've probably thrown out, as William pointed out on sage-devel) XCode anyway, in which case we might as well only worry about Lion anyway.
Or would this eliminate having Xcode as a prerequisite to building Sage on OS X in general (even for those for which XCode is free, though sometimes annoying to get)? That might be a nice move. I assume this is a prebuilt gcc like our current Fortran package?
No, this is a gcc from source. I certainly don't want to include gcc binaries for every platform, that would increase the size way too much.
That's what I figured, but just wanted to clarify.
Could the message "installing GCC because..." occur several times here?
Sure.
Ok. I suppose it doesn't really matter, but could be confusing/alarming.
Replying to @kcrisman:
Possibly. I guess what I'm wondering is how one would build this gcc if one didn't have gcc... but this may betray my ignorance.
My hope is that the non-gcc C compiler shipped with XCode 4 might be able to build GCC.
Anyway, the vast majority of Mac users will not have a way to build this gcc if they need gcc until they download/install
That "vast majority" of users will be happy with a binary distribution of Sage (which could actually contain GCC if I manage to make this spkg work).
Replying to @jdemeyer:
Replying to @kcrisman:
Possibly. I guess what I'm wondering is how one would build this gcc if one didn't have gcc... but this may betray my ignorance.
My hope is that the non-gcc C compiler shipped with XCode 4 might be able to build GCC.
Interesting! Now I see the point of all this.
Anyway, the vast majority of Mac users will not have a way to build this gcc if they need gcc until they download/install
That "vast majority" of users will be happy with a binary distribution of Sage
Good point - until they need to change any code. But this
(which could actually contain GCC if I manage to make this spkg work).
Interesting, as I said. I wonder if this would work:
Changed dependencies from #11073, #12366, #12367, #12368, #10492, #12405 to #11073, #12366, #12367, #12368, #10492, #12405, #12416
Changed dependencies from #11073, #12366, #12367, #12368, #10492, #12405, #12416 to #11073, #12366, #12367, #12368, #10492, #12405, #12416, #12422
Changed dependencies from #11073, #12366, #12367, #12368, #10492, #12405, #12416, #12422 to #11073, #10492, #12366, #12367, #12368, #12405, #12416, #12422, #12423
Changed dependencies from #11073, #10492, #12366, #12367, #12368, #12405, #12416, #12422, #12423 to #11073, #10492, #12366, #12367, #12368, #12405, #12416, #12422, #12423, #12425
I haven't looked at the spkg yet (which is probably old anyway), but looking at the root patch that you wrote, I have a few of requests:
I agree with Karl. Do not install gcc if gcc is new enough (>=4.0.1), this will cause every mac developer to have to build gcc upon building sage (significantly increasing the build time). I'm not a mac user, but I know that if I had to do this on my system, I would be annoyed.
Assume the user knows what they are doing it CC is set (like you do for CXX and SAGE_FORTRAN). While not a very common use case, it would be helpful to not have to go hack the installation file if you were trying to port sage to another compiler (like I am right now).
(You might be doing this already) If fortran is the only language that needs to be built (such as on macs), please do not build gcc and g++.
And a couple comments:
You assume that CC supports -dumpversion when determining if you should upgrade gcc. I'm fairly certain this is not an option supported by all compilers. Also, the value you would get would be meaningless anyway, since it wouldn't be a gcc version anyway (well, unless it is clang).
For OBJC and OBJCXX, I would point to /usr/bin/cc and /usr/bin/cpp instead. If/when apple stops distributing gcc/g++, then these will continue to point to valid compilers.
In response to Karl about a replacement for Xcode (or tools on any platform). This does not provide one, even in binary distributions of sage since we need binutils and make, neither of which this would provide (and neither of which are on OSX without Xcode).
In response to Karl about a replacement for Xcode (or tools on any platform). This does not provide one, even in binary distributions of sage since we need binutils and make, neither of which this would provide (and neither of which are on OSX without Xcode).
Right, that makes sense. I didn't know exactly what comes from Xcode and what is just pre-included.
I've been wondering for a while if we should change Sage to use autoconf. (When I say "we", I don't actually mean me, since I don't know anything about writing configure
scripts, nor do I want to learn.) This could replace spkg-install (especially with the modifications at #10492 that create a Makefile on the fly). Does this make sense as an enhancement? Should I poll sage-devel about it, and then open a ticket if the response is positive?
Replying to @jhpalmieri:
I've been wondering for a while if we should change Sage to use autoconf. (When I say "we", I don't actually mean me, since I don't know anything about writing
configure
scripts, nor do I want to learn.) This could replace spkg-install (especially with the modifications at #10492 that create a Makefile on the fly). Does this make sense as an enhancement?
I don't know how much additional functionality it would add, but I would argue for it since it is more unix user friendly than our current custom setup (SAGE_ATLAS_LIB is not as user friendly as a fairly standard --with-system-atlas argument for configure). My guess is that it would be a fairly large endeavour to implement a first version of such a change (I don't know autoconf, so I'm just guessing here), although it would hopefully be easier to maintain than spkg-install.
Should I poll sage-devel about it, and then open a ticket if the response is positive?
I don't think it could hurt. Also, that would be a better place to discuss it than on this ticket.
Replying to @ohanar:
- Assume the user knows what they are doing it CC is set (like you do for CXX and SAGE_FORTRAN). While not a very common use case, it would be helpful to not have to go hack the installation file if you were trying to port sage to another compiler (like I am right now).
Actually, thinking a bit more about it, I think it would be better if we required SAGE_PORT to be set if CC and/or CXX is set.
Replying to @ohanar:
I haven't looked at the spkg yet (which is probably old anyway), but looking at the root patch that you wrote, I have a few of requests:
- I agree with Karl. Do not install gcc if gcc is new enough (>=4.0.1), this will cause every mac developer to have to build gcc upon building sage (significantly increasing the build time). I'm not a mac user, but I know that if I had to do this on my system, I would be annoyed.
Well, we do need to build Fortran on a Mac (I'm assuming that the fortran spkg will go away). And Sage seems to require that the gcc and gfortran versions are the same. It is true that the build time of gcc is significant:
real 21m4.603s
user 15m56.183s
sys 4m26.492s
Successfully installed gcc-4.4.6
(on bsd.math, single-threaded), but it's not that bad. I think the "plus" of having a working Sage compiled with optimizations (as opposed to using XCode 4 with many packages compiled with -O0) outweighs the "minus" of 25 minutes longer build time.
- Assume the user knows what they are doing it CC is set (like you do for CXX and SAGE_FORTRAN). While not a very common use case, it would be helpful to not have to go hack the installation file if you were trying to port sage to another compiler (like I am right now).
Well, I was thinking anyway of adding an environment variable (or future ./configure
option) to either force installation of gcc, or disable installation of gcc. Don't forget that GCC has a few dependencies, and one might set CC
to build those dependencies, but still want GCC to be installed. Anyway, I don't care much about this.
- (You might be doing this already) If fortran is the only language that needs to be built (such as on macs), please do not build gcc and g++.
I thought gcc and gfortran needed to be the same versions?
And a couple comments:
- You assume that CC supports -dumpversion when determining if you should upgrade gcc. I'm fairly certain this is not an option supported by all compilers. Also, the value you would get would be meaningless anyway, since it wouldn't be a gcc version anyway (well, unless it is clang).
Agreed.
- For OBJC and OBJCXX, I would point to /usr/bin/cc and /usr/bin/cpp instead. If/when apple stops distributing gcc/g++, then these will continue to point to valid compilers.
Agreed (modulo the fact that you mean c++
instead of the preprocessor cpp
).
In response to Karl about a replacement for Xcode (or tools on any platform). This does not provide one, even in binary distributions of sage since we need binutils and make, neither of which this would provide (and neither of which are on OSX without Xcode).
True...
Replying to @jdemeyer:
Well, we do need to build Fortran on a Mac (I'm assuming that the fortran spkg will go away).
Agreed, which is why I would like only fortran to be built if it is the only language needed.
I think the "plus" of having a working Sage compiled with optimizations (as opposed to using XCode 4 with many packages compiled with -O0) outweighs the "minus" of 25 minutes longer build time.
I think this is something that should be brought to the listserv. Once I get a clang port up and running (which shouldn't be too bad on osx, since atlas looks like it will be the only package that is going to be hell to port), this can be resolved by changing the default compiler on osx to clang.
- (You might be doing this already) If fortran is the only language that needs to be built (such as on macs), please do not build gcc and g++.
I thought gcc and gfortran needed to be the same versions?
I'm not sure about this, I'm using a mixture of clang (which dumps its version as 4.2.1) and gfortran-4.5.3, and occasionally gcc-4.2.4 and g++-4.2.4 to build sage, and currently the entire test suite is passing.
Replying to @ohanar:
Agreed, which is why I would like only fortran to be built if it is the only language needed.
Is build time your only concern here? Because if you need to build Fortran, you most likely need to build a stage 1 C compiler anyway.
I would like to add that my goal is beyond OS X 10.7 but more generally all systems with bad compilers. That's also why I went for the more stable gcc-4.4.6 as opposed to the more recent (and therefore buggy) gcc-4.6.2.
Replying to @jdemeyer:
I think the "plus" of having a working Sage compiled with optimizations (as opposed to using XCode 4 with many packages compiled with -O0) outweighs the "minus" of 25 minutes longer build time.
To be fair, the current situation requires only one package (symmetrica) to be compiled without optimization, and then only if the extra gcc-4.2 package is not present.
Replying to @jdemeyer:
I would like to add that my goal is beyond OS X 10.7 but more generally all systems with bad compilers.
I have compiled gcc too often on clusters that have only old versions installed.
Personally, gcc-4.6.2 seems to work very well. I'm not saying we should jump on every .0, mind you ;-)
Replying to @jdemeyer:
Is build time your only concern here? Because if you need to build Fortran, you most likely need to build a stage 1 C compiler anyway.
Yes, I guess that is true, there is probably minimal savings by not building c++. I guess then as long as we don't add this to the critical path for most systems it should be fine. Do you have a current spkg that I could look at? I would like to see if there is any quick ways to reduce build time (e.g. disabling multilib on 64-bit systems).
Replying to @vbraun:
Personally, gcc-4.6.2 seems to work very well. I'm not saying we should jump on every .0, mind you ;-)
Same here (plus it compiles with clang, unlike 4.4.6, from my experimenting), however it adds mpc as a dependency.
Description changed:
---
+++
@@ -3,3 +3,5 @@
**spkg**: [http://boxen.math.washington.edu/home/jdemeyer/spkg/gcc-4.4.6.spkg](http://boxen.math.washington.edu/home/jdemeyer/spkg/gcc-4.4.6.spkg)
**Apply** [attachment: 12369_gcc_root.patch](https://github.com/sagemath/sage-prod/files/10654635/12369_gcc_root.patch.gz) to the `SAGE_ROOT` repository.
+
+**Testing release**: [http://boxen.math.washington.edu/home/jdemeyer/release/sage-5.0.beta1-gcc/](http://boxen.math.washington.edu/home/jdemeyer/release/sage-5.0.beta1-gcc/)
Replying to @ohanar:
Yes, I guess that is true, there is probably minimal savings by not building c++. I guess then as long as we don't add this to the critical path for most systems it should be fine. Do you have a current spkg that I could look at?
The spkg on this ticket is current.
Replying to @ohanar:
I guess then as long as we don't add this to the critical path for most systems it should be fine.
If it gets built, it will be on the critical path (since it effectively blocks the build). However, that's not a problem since the GCC build paralellizes well: I managed to build it in 4 minutes using "make -j24":
real 4m7.170s
user 20m58.059s
sys 4m29.269s
Successfully installed gcc-4.4.6
Replying to @ohanar:
any quick ways to reduce build time (e.g. disabling multilib on 64-bit systems).
My gcc-4.6.2 spkg disables multilib already
Same here (plus it compiles with clang, unlike 4.4.6, from my experimenting), however it adds mpc as a dependency.
For the record, mpc is contained in the gcc-4.6.2 spkg.
I posted a patch for the gcc-4.4.6 spkg, it disables multilib, and fixes clang build (it also disables warnings, since gcc throws so many of them, I actually noticed a small speed up from disabling them).
Replying to @jdemeyer:
If it gets built, it will be on the critical path (since it effectively blocks the build).
But you also need to consider that we are building the mpir and mpfr twice as well as adding one copy of those builds to the critical path. I still think there is value to distinguishing between when gcc is needed, and when only fortran is needed. In the latter case, this should actually negligibly shorten the critical path (with the removal of the fortran spkg) plus mpir+mpfr don't have to be rebuilt.
My concern is that sage already takes a good while to build, and this spkg adds a fairly significant time to that for only a few feature gains and relatively minor performance gains (compared to implementing new/better algorithms).
By the way, I know you weren't using the word "path" this way, but regarding the path, note for example #10822: on some platforms, apparently, we may not be able to guarantee that SAGE_ROOT/local/bin
comes first in the path. So in the patch for the root repo, would it better to replace
CC=gcc
by
CC="$SAGE_LOCAL/bin/gcc"
on line 418 of sage-env, and similarly for CXX?
My personal opinion on the build time is the following: I don't care much about it. I much rather have a Sage which works and is fast than a Sage which builds fast.
There is no need to rebuild mpir and mpfr, except for speed gains (the newer GCC's might have better support for newer processors).
Replying to @jhpalmieri:
CC="$SAGE_LOCAL/bin/gcc"
on line 418 of sage-env, and similarly for CXX?
Unfortunately, this would break if "$SAGE_LOCAL" contains spaces or other weird characters since $CC is usually used unquoted.
I believe the zsh "sage -sh" issues are beyond gcc (it totally breaks everything) and should be fixed on #10822, not here.
Changed dependencies from #11073, #10492, #12366, #12367, #12368, #12405, #12416, #12422, #12423, #12425 to #11073, #10492, #12366, #12367, #12368, #12405, #12416, #12422, #12423, #12425, #12456, #12457
Changed dependencies from #11073, #10492, #12366, #12367, #12368, #12405, #12416, #12422, #12423, #12425, #12456, #12457 to #11073, #10492, #12366, #12367, #12368, #12405, #12416, #12422, #12423, #12425, #12456, #12457, #12459
Why is -std=c89
needed?
Replying to @jdemeyer:
Why is
-std=c89
needed?
-std=gnu89
is not needed for gcc, but is needed for clang, and since I'm working on porting sage to clang, I would rather have this package work on it out of the gate (rather than make a ticket for something that could just be fixed now). Alternatively, we could use a more recent version of gcc in the spkg, since, at the very least, gcc-4.6.2 uses gnu99, the default standard for clang.
I would also suggest using making cc and c++ be the default compiler when building gcc and its dependencies. Like OBJC and OBJCXX, this is more future thinking when it comes to apple platforms (plus more forward thinking for any platform with an unsupported compiler, from which we can build gcc).
Changed dependencies from #11073, #10492, #12366, #12367, #12368, #12405, #12416, #12422, #12423, #12425, #12456, #12457, #12459 to #11073, #10492, #12366, #12367, #12368, #12405, #12416, #12422, #12423, #12425, #12456, #12457
Volker, I just had a quick look at your spkg:
Description changed:
---
+++
@@ -1,6 +1,6 @@
The aim is to add a gcc package to Sage. We probably don't want to always build it, we would install it by default only on systems where this is needed. This would replace the fortran package.
-**spkg**: [http://boxen.math.washington.edu/home/jdemeyer/spkg/gcc-4.4.6.spkg](http://boxen.math.washington.edu/home/jdemeyer/spkg/gcc-4.4.6.spkg)
+**spkg**: [http://boxen.math.washington.edu/home/jdemeyer/spkg/gcc-4.6.2.spkg](http://boxen.math.washington.edu/home/jdemeyer/spkg/gcc-4.6.2.spkg)
**Apply** [attachment: 12369_gcc_root.patch](https://github.com/sagemath/sage-prod/files/10654635/12369_gcc_root.patch.gz) to the `SAGE_ROOT` repository.
Wow, compiling this spkg with SAGE_CHECK=yes
is pretty painful: on my Mac (which has only two cores), it took 89 minutes.
Replying to @jhpalmieri:
Wow, compiling this spkg with
SAGE_CHECK=yes
is pretty painful: on my Mac (which has only two cores), it took 89 minutes.
Note that compiling GCC with SAGE_CHECK=yes
will actually make every spkg build slower, because it enables run-time checks inside GCC.
I think this is not a problem and fits with the spirit of SAGE_CHECK=yes
.
Yes, the gcc testsuite is pretty big!
Replying to @vbraun:
Yes, the gcc testsuite is pretty big!
I'm not actually shipping nor running the gcc testsuite. It would add 6MB to the spkg size.
I've built the "testing release" on OS X Lion with SAGE_CHECK=yes
. The results:
make check-TESTS
PASS: ascii_dump_load1
/bin/sh: line 1: 22675 Abort trap: 6 ${dir}$tst
FAIL: exceptions1
PASS: pipproblem1
PASS: pipproblem2
PASS: pipproblem3
make
on Sage with MAKE="make -j2"
so that other compilations are going on at the same time, but it often passes if I just do ./sage -i spkg/standard/zn_poly...
.Doctests haven't finished yet.
PARI doesn't work with gcc-4.6.2 on OS X 10.6 and 10.7 systems. Luckily, the new PARI (#12363) fixes this.
Changed dependencies from #11073, #10492, #12366, #12367, #12368, #12405, #12416, #12422, #12423, #12425, #12456, #12457 to #11073, #10492, #12366, #12367, #12368, #12405, #12416, #12422, #12423, #12425, #12456, #12457, #12363
The aim is to add a GCC package (GNU compiler collection) to Sage with compilers for C, C++ and Fortran. We don't always build it, we would install it by default only on systems where this is needed. This would replace the fortran package.
spkg: http://boxen.math.washington.edu/home/jdemeyer/spkg/gcc-4.6.3.spkg
Apply:
SAGE_ROOT
repository.SCRIPTS
repository.See also:
12782: When building GCC, build MPIR without the C++ interface (superseded by #11616).
Depends on #12479 Depends on #12602 Depends on #12608 Depends on #12609 Depends on #12647 Depends on #10492 Depends on #12367 Depends on #12368 Depends on #12405 Depends on #12570 Depends on #12574 Depends on #12423 Depends on #12425 Depends on #12456 Depends on #12363 Depends on #12223 Depends on #12515 Depends on #12519 Depends on #12548 Depends on #12562 Depends on #12629 Depends on #12638 Depends on #12714 Depends on #12647 Depends on #12739 Depends on #12112 Depends on #12631
CC: @vbraun
Component: packages: standard
Author: Jeroen Demeyer
Reviewer: Simon King
Merged: sage-5.0.beta13
Issue created by migration from https://trac.sagemath.org/ticket/12369