search

sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.37k stars 468 forks source link

Meta-ticket: allow more Fortran compilers #23926

Open dimpase opened 7 years ago

dimpase commented 7 years ago

There are Fortran compilers which give better than gfortran performance, e.g. Intel's ifort. It is currently not possible (without a hack) to build Sage using any of these.

This ought to be fixed.

See also:

Discussions:

CC: @kiwifb @isuruf @slel

Component: build

Issue created by migration from https://trac.sagemath.org/ticket/23926

dimpase commented 6 years ago
comment:2

flang -- the LLVM-based Fortan compiler looks very promising. Besides being open-source, it is already packaged on FreeBSD, probably meaning that people are able to do serious work using it. (It is also in Conda, meaning that we are sort of covered on OSX, too).

I am able to build Sage using the system-provided (pkg install flang) flang on FreeBSD, by removing need_to_install_gfortran=yes from configure.ac, running configure with ./configure CC=cc CXX=c++ FC=flang (that is, the whole toolchain is LLVM-based).

scipy needed a minor fix, as it hardcodes not needed libgfortran, as well as apparently gfortran too, so I tricked it by making symbolic links libgfortan.so to point to libz.so (something innocent) and gfortran to point to flang. (The latter could be replaced by export FC=flang).

Also, r needed a fix, it was unable to determine what should be in FPICFLAGS, and so I did FPICFLAGS=-fPIC ./sage -f r.

isuruf commented 6 years ago
comment:4

Replying to @dimpase:

flang -- the LLVM-based Fortan compiler looks very promising. Besides being open-source, it is already packaged on FreeBSD, probably meaning that people are able to do serious work using it. (It is also in Conda, meaning that we are sort of covered on OSX, too).

We package flang for only Linux and Windows on conda. OSX is still not supprted, but I don't think it would take much time given that flang can be used on FreeBSD. (Windows support is also by a fork we maintain until upstream decides on a CLA which seems to be taking a lot of time.)

I am able to build Sage using the system-provided (pkg install flang) flang on FreeBSD, by removing need_to_install_gfortran=yes from configure.ac, running configure with ./configure CC=cc CXX=c++ FC=flang (that is, the whole toolchain is LLVM-based).

scipy needed a minor fix, as it hardcodes not needed libgfortran, as well as apparently gfortran too, so I tricked it by making symbolic links libgfortan.so to point to libz.so (something innocent) and gfortran to point to flang. (The latter could be replaced by export FC=flang).

numpy distutils PR is at https://github.com/numpy/numpy/pull/9942

dimpase commented 6 years ago
comment:5

Replying to @isuruf:

Replying to @dimpase:

flang -- the LLVM-based Fortan compiler looks very promising. Besides being open-source, it is already packaged on FreeBSD, probably meaning that people are able to do serious work using it. (It is also in Conda, meaning that we are sort of covered on OSX, too).

We package flang for only Linux and Windows on conda. OSX is still not supprted, but I don't think it would take much time given that flang can be used on FreeBSD. (Windows support is also by a fork we maintain until upstream decides on a CLA which seems to be taking a lot of time.)

By the way, is there any chance that FreeBSD (or perhaps DragonflyBSD, which is also clang-based w.r.t. the toolchain) gets properly supported by Conda (i.e. in more or less the same way as Linux and OSX are)?

I am able to build Sage using the system-provided (pkg install flang) flang on FreeBSD, by removing need_to_install_gfortran=yes from configure.ac, running configure with ./configure CC=cc CXX=c++ FC=flang (that is, the whole toolchain is LLVM-based).

scipy needed a minor fix, as it hardcodes not needed libgfortran, as well as apparently gfortran too, so I tricked it by making symbolic links libgfortan.so to point to libz.so (something innocent) and gfortran to point to flang. (The latter could be replaced by export FC=flang).

numpy distutils PR is at https://github.com/numpy/numpy/pull/9942

Apart from some linking issues in src/sage/misc/inline_fortran.py (somewhere -lgfortran hardcoded, etc), I see little to no difference with Sage tests using gfortran.

kiwifb commented 6 years ago
comment:6

Out of interest, does autoconf think that flang is gfortran? In other words, does it put GFC to yes?

dimpase commented 6 years ago
comment:7

Replying to @kiwifb:

Out of interest, does autoconf think that flang is gfortran? In other words, does it put GFC to yes?

If I omit

--- a/configure.ac
+++ b/configure.ac
@@ -466,8 +466,6 @@ elif test $need_to_install_gcc = yes; then
     true
 elif test x$GCC != xyes; then
     SAGE_SHOULD_INSTALL_GCC([your C compiler isn't GCC (GNU C)])
-elif test x$GFC != xyes; then
-    need_to_install_gfortran=yes
 else
     # Since need_to_install_gcc is "no", we know that
     # at least C, C++ and Fortran compilers are available.

then an attempt to install gfortran package is made.

isuruf commented 6 years ago
comment:8

Replying to @dimpase:

By the way, is there any chance that FreeBSD (or perhaps DragonflyBSD, which is also clang-based w.r.t. the toolchain) gets properly supported by Conda (i.e. in more or less the same way as Linux and OSX are)?

I don't think Anaconda, Inc ever will unless there is a paying customer.

That said there are some people maintaining conda packages for other non-supported platforms like https://github.com/jjhelmus/berryconda

If you want to build conda packages in FreeBSD, conda-forge recipes are a good place. Another option would be to create a conda package from the sage package like I used to do. (I abandoned it because it wouldn't be compatible with the rest of conda packages on other channels, but in the FreeBSD case, that shouldn't matter)

dimpase commented 6 years ago
comment:9

Replying to @isuruf:

Replying to @dimpase:

flang -- the LLVM-based Fortan compiler looks very promising. Besides being open-source, it is already packaged on FreeBSD, probably meaning that people are able to do serious work using it. (It is also in Conda, meaning that we are sort of covered on OSX, too).

We package flang for only Linux and Windows on conda. OSX is still not supprted, but I don't think it would take much time given that flang can be used on FreeBSD. (Windows support is also by a fork we maintain until upstream decides on a CLA which seems to be taking a lot of time.)

Yeah, there seem to be no flang on OSX available as yet. Flang on FreeBSD is an out-of-tree branch ATM, too. So this is all in a flux now.

I guess it might be wise to wait until flang can be built with unmodified clang...

dimpase commented 5 years ago
comment:10

Just for giggles and lols, I tried building Sage 8.7.beta0 using flang on FreeBSD 12.0, and it just works!

Well, it needs various FFLAGS and LDFLAGS to be passed, which is probably more of a FreeBSD bug than anything else, and I didn't trace down why scipy tries to link with libgfortran.so (I ended up just faking libgfortran.so, making it a link to libm.so :-))

But the fact that the configuration just works otherwise, means that this ticket is almost done, all of a sudden!

The quality of flang leaves much to be desired especially in complex domain arithmetic, at least openblas tests and Sage tests show a number of errors, but, OK, it is probably fixable on the flang side...

embray commented 4 years ago
comment:12

Ticket retargeted after milestone closed

mkoeppe commented 4 years ago
comment:13

Batch modifying tickets that will likely not be ready for 9.1, based on a review of the ticket title, branch/review status, and last modification date.

mkoeppe commented 3 years ago
comment:15

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.

mkoeppe commented 3 years ago

Description changed:

--- 
+++ 
@@ -2,3 +2,8 @@
 It is currently not possible (without a hack) to build Sage using any of these.

 This ought to be fixed. 
+
+See also: 
+- #32176 Support building Sage with the Flang Fortran compiler
+- #32177 Support building Sage with the LFortran Fortran compiler
+
mkoeppe commented 3 years ago

Description changed:

--- 
+++ 
@@ -7,3 +7,6 @@
 - #32176 Support building Sage with the Flang Fortran compiler
 - #32177 Support building Sage with the LFortran Fortran compiler

+Discussions:
+- https://groups.google.com/g/sage-devel/c/NfUKjAaTcUg/m/9-wV7jigAwAJ (François Bissey, Sep 2021)
+
dimpase commented 3 years ago
comment:18

I cannot figure out how to pass the fortran compiler to pip while building SciPy and NumPy. Yes, I know one can do python setup.py build --fcompiler=$FC, assuming FC holds the compiler, but how to combine this with installation by pip3?

mkoeppe commented 3 years ago
comment:19

numpy's spkg-install.in knows how

dimpase commented 3 years ago
comment:20

Replying to @mkoeppe:

numpy's spkg-install.in knows how

Does it? How does one tell config_fc which Fortran compiler one wants?

Another "interesting" story there is how does one tell config_fc what to do with Fortran files which need a pre-processor? There are Fortran compilers that don't do preprocessing itself, e.g. nagfor from NAG doesn't understand -D and -U options one might need to run on .F files (with C-style #ifdefs); it comes with an fpp utility that is supposed to be run on such files. See https://github.com/scipy/scipy/issues/13222#issuecomment-927962803