Remove gcc spkg, keep gfortran

[dimpase]

Discussions:

https://groups.google.com/g/sage-devel/c/cSsAsPuVnxg/m/yVSGUFi7BQAJ (June 2021) https://groups.google.com/g/sage-devel/c/NfUKjAaTcUg/m/JwqDd7mVAwAJ (September 2021) In this ticket:

Just downgrade the gcc spkg to experimental, to match our recommendations in the installation guide in #34528. CC: @mkoeppe @vbraun @orlitzky @jhpalmieri

Component: packages: standard

Author: Dima Pasechnik

Branch/Commit: u/dimpase/packages/gcc_experimental @ 58ca5d2

Issue created by migration from https://trac.sagemath.org/ticket/32532

(NB: reverted to the original state; copuous edits by @mkoeppe, done in 2024 are still in history)

[dimpase]

comment:1

Dima's original ticket description:

This is a followup to #32060. Building gcc from source is very hard, and unnecessary for any platform Sage supports now.

Same applies to gfortran, with macOS being an exception - there is no (g)fortran coming from the system.

So we need such an easy to install replacement, e.g. one provided by Conda, Homebrew, MacPorts, etc.

Should the need arise for something more isolated, (hopefully not), there are experimental standalone distributions of gfortran for macOS, by the gfortran maintainer of the Homebrew package. This offers a way to entirety remove gcc and gfortran spkgs from Sage.

​https://github.com/fxcoudert/gfortran-for-macOS - specifically, https://github.com/fxcoudert/gfortran-for-macOS/releases/tag/11.2-bigsur-intel provides a signed and notarized installed of gcc 11.1 to /usr/local (I tested it, it works)

Other Fortran possibilities are various closed source offers, e.g. from Intel and NAG, as well as open-source flang, etc.

[dimpase]

Description changed:

--- 
+++ 
@@ -4,7 +4,7 @@

 So we need such an easy to install replacement (as opposed to ones provided by Conda, Homebrew, MacPorts, etc. which come as a part of a complete distro).

-There are standalone distributions of gfortran for macOS, by the gfortran maintainer of the Homebrew package. To me this offers a way to entirety remove gcc and gfortran spkgs from Sage.
+There are standalone distributions of gfortran for macOS, by the gfortran maintainer of the Homebrew package. This offers a way to entirety remove gcc and gfortran spkgs from Sage.

 ​https://github.com/fxcoudert/gfortran-for-macOS

[mkoeppe]

comment:3

There are no build problems with gfortran - #32060 comment:36

[dimpase]

comment:5

yet, building gfortran is useless waste of resources - it takes a lot of time, etc.

[mkoeppe]

comment:6

Removing it puts a fresh obstacle in the way of pip-installability -- no, thanks.

[jhpalmieri]

comment:7

Since removing gfortran is controversial, please drop the idea or discuss on sage-devel. It should not be settled here.

[dimpase]

comment:8

Replying to @mkoeppe:

Removing it puts a fresh obstacle in the way of pip-installability -- no, thanks.

how come removing gcc spkg does not create such an obstacle, whereas it does for gfortran?

Besides, if you need to build gcc in order to build gfortran, the latter cannot be counted as unproblematic.

[dimpase]

comment:9

still needs adding some Fortran to CI for the minimal macOS configuration.

---

Last 10 new commits:

48cdeb4	configure.ac, sage_spkg_configure.m4: Set default --with-system-gcc=force, --with-system-python3=force
8e92b98	tox.ini: Override new configure --with-system-python3=force default from #32060
19efe21	configure.ac: Revert the change to the default for --with-system-python3
c5e1dec	add gfortran to _prereq/distros
2482aa1	error out if C/C++ is not good enough
6366b2a	error out if no good Fortran found.
f103f2a	remove *gcc_spkg workflows
f58e236	correct the order of messages
657c1e0	adjust tox.ini for no gcc spkg
5461d81	correct order of messages

[dimpase]

Branch: u/dimpase/packages/removegcc

[dimpase]

Commit: 5461d81

[dimpase]

comment:10

also, CI tests for obsolete gcc-4 based distros are gone, about time.

[mkoeppe]

comment:11

Replying to @dimpase:

how come removing gcc spkg does not create such an obstacle, whereas it does for gfortran?

Python dev environments have a gcc. If they don't, we can't help them anyway.

Python dev environments don't have gfortran. gfortran is not pip-installable. The few Python packages that use gfortran have high-quality deployments of wheels to PyPI, which are widely used.

Sage does not (yet?) have a working strategy that would enable us to use wheels from PyPI. The first step to pip-installability in #29039 builds everything from source.

Besides, if you need to build gcc in order to build gfortran, the latter cannot be counted as unproblematic.

Come again? No, we don't do that on any platform.

[dimpase]

comment:12

Replying to @mkoeppe:

Replying to @dimpase:

how come removing gcc spkg does not create such an obstacle, whereas it does for gfortran?

Python dev environments have a gcc. If they don't, we can't help them anyway.

Python dev environments don't have gfortran. gfortran is not pip-installable.

gcc is not pip-installable, either. The only scenario where developers have no Fortran, but "gcc" (for some value of "gcc", e.g. Apple clang) is on macOS, as far as I know. On macOS they'd have to help themselves by installing (g, or some other)fortran in some way.

Needless to say, we should test that this is feasible without using !Homebrew/Conda/macports, say with the gfortran installer linked in the ticket description.

The few Python packages that use gfortran have high-quality deployments of wheels to PyPI, which are widely used.

Sage does not (yet?) have a working strategy that would enable us to use wheels from PyPI. The first step to pip-installability in #29039 builds everything from source.

OK, and this setting will need a Fortran compiler in addition to a C/C++ compiler. Why is it an extra obstacle? It's an improvement, as it makes building a lot quicker.

Besides, if you need to build gcc in order to build gfortran, the latter cannot be counted as unproblematic.

Come again? No, we don't do that on any platform.

on gcc-4-based platforms we build gcc+gfortran, no? (well, I am not worried about this obsolete case)

[mkoeppe]

comment:13

Replying to @dimpase:

Replying to @mkoeppe:

Replying to @dimpase:

how come removing gcc spkg does not create such an obstacle, whereas it does for gfortran?

Python dev environments have a gcc. If they don't, we can't help them anyway.

Python dev environments don't have gfortran. gfortran is not pip-installable.

gcc is not pip-installable, either.

Please... I just explained it.

"Python dev environments have a gcc". Why? Because lots of Python packages need a working C/C++ compiler.

I explained why the gfortran situation is different.

[dimpase]

comment:14

Replying to @mkoeppe:

Replying to @dimpase:

Replying to @mkoeppe:

Replying to @dimpase:

how come removing gcc spkg does not create such an obstacle, whereas it does for gfortran?

Python dev environments have a gcc. If they don't, we can't help them anyway.

Python dev environments don't have gfortran. gfortran is not pip-installable.

gcc is not pip-installable, either.

Please... I just explained it.

"Python dev environments have a gcc". Why? Because lots of Python packages need a working C/C++ compiler.

I explained why the gfortran situation is different.

It's not different, as Fortran compilers are easy to come by these days.

The philosophy of this ticket is that we should cut our losses we had while trying to vendor everything imaginable, spending human resources on thankless tasks such as vendoring gcc/gfortran, something that 99.9% of Sage users don't care about.

[mkoeppe]

comment:15

-1 on creating new real problems in order to fix imaginary problems.

[dimpase]

comment:16

Replying to @mkoeppe:

-1 on creating new real problems in order to fix imaginary problems.

No Python project I know vendors compilers. Lack of developer time is very real problem, and you keep perpetuating this problem by refusing to shed the ballast.

[dimpase]

Author: Dima Pasechnik

[dimpase]

comment:17

I did a run on GitHub Actions: https://github.com/dimpase/sagetrac-mirror/actions/runs/1258945033 (our macOS actions are broken, so no surprise it's not working there)

[dimpase]

Description changed:

--- 
+++ 
@@ -2,12 +2,12 @@

 Same applies to gfortran, with macOS being an exception - there is no (g)fortran coming from the system.

-So we need such an easy to install replacement (as opposed to ones provided by Conda, Homebrew, MacPorts, etc. which come as a part of a complete distro).
+So we need such an easy to install replacement, e.g. one provided by Conda, Homebrew, MacPorts, etc.

-There are standalone distributions of gfortran for macOS, by the gfortran maintainer of the Homebrew package. This offers a way to entirety remove gcc and gfortran spkgs from Sage.
+Should the need arise for something more isolated, (hopefully not), there are experimental standalone distributions of gfortran for macOS, by the gfortran maintainer of the Homebrew package. This offers a way to entirety remove gcc and gfortran spkgs from Sage. 

 ​https://github.com/fxcoudert/gfortran-for-macOS

-other Fortran possibilities are various closed source offers, e.g. from Intel and NAG,
+Other Fortran possibilities are various closed source offers, e.g. from Intel and NAG,
 as well as open-source flang, etc.

[mkoeppe]

comment:18

Let's please drop this.

[dimpase]

comment:19

you have not explained why it should be dropped, and what kind of difficulty it'd present if it were merged.

[jhpalmieri]

comment:21

Take the discussion to sage-devel.

[mkoeppe]

Description changed:

--- 
+++ 
@@ -11,3 +11,10 @@
 Other Fortran possibilities are various closed source offers, e.g. from Intel and NAG,
 as well as open-source flang, etc.

+Discussions:
+
+- https://groups.google.com/g/sage-devel/c/cSsAsPuVnxg/m/yVSGUFi7BQAJ (June 2021)
+- https://groups.google.com/g/sage-devel/c/NfUKjAaTcUg/m/JwqDd7mVAwAJ (September 2021)
+
+
+

[dimpase]

Description changed:

--- 
+++ 
@@ -6,7 +6,10 @@

 Should the need arise for something more isolated, (hopefully not), there are experimental standalone distributions of gfortran for macOS, by the gfortran maintainer of the Homebrew package. This offers a way to entirety remove gcc and gfortran spkgs from Sage. 

-​https://github.com/fxcoudert/gfortran-for-macOS
+​https://github.com/fxcoudert/gfortran-for-macOS - specifically, 
+https://github.com/fxcoudert/gfortran-for-macOS/releases/tag/11.2-bigsur-intel
+provides a signed and notarized installed of gcc 11.1 to ./usr/local (I tested it, it works)
+

 Other Fortran possibilities are various closed source offers, e.g. from Intel and NAG,
 as well as open-source flang, etc.

[dimpase]

Description changed:

--- 
+++ 
@@ -8,7 +8,7 @@

 ​https://github.com/fxcoudert/gfortran-for-macOS - specifically, 
 https://github.com/fxcoudert/gfortran-for-macOS/releases/tag/11.2-bigsur-intel
-provides a signed and notarized installed of gcc 11.1 to ./usr/local (I tested it, it works)
+provides a signed and notarized installed of gcc 11.1 to `/usr/local` (I tested it, it works)

 Other Fortran possibilities are various closed source offers, e.g. from Intel and NAG,

[dimpase]

comment:25

given that M1 cannot use Sage's gcc/gfortran, perhaps it's high time to move here?

[mkoeppe]

Changed commit from 5461d81 to none

[mkoeppe]

comment:26

Time for "wontfix"

[mkoeppe]

Changed branch from u/dimpase/packages/removegcc to none

[mkoeppe]

Changed author from Dima Pasechnik to none

[dimpase]

comment:27

Huh? If we support platforms on which a package doesn't work, it can't be standard,or even optional.

either go forward with this ticket and remove it,or make it experimental.

[jhpalmieri]

comment:28

33816 allows the gfortran package to build on M1/M2.

[jhpalmieri]

comment:29

As far as OS X is concerned, we could drop the gcc package. I don't know if that would create issues on other platforms, but we could ask on sage-devel.

[mkoeppe]

comment:30

I think it makes sense to downgrade the gcc package to "experimental", which would reflect the fact that we no longer test gcc_spkg configurations on our CI.

[jhpalmieri]

comment:31

For what it's worth, it took about 10 minutes for gfortran to build on two different machines, one m2 (9 minutes), one Intel (11 minutes). So it's long but faster than sagelib and not much slower than scipy. It's much faster than building Sage's gcc used to be, if I remember correctly.

[dimpase]

comment:32

Replying to Matthias Köppe:

I think it makes sense to downgrade the gcc package to "experimental", which would reflect the fact that we no longer test gcc_spkg configurations on our CI.

downgrading to experimental means inevitable bitrot - happening already, right? If noone uses it in practice, I see no point in keeping it, however quick it can be built.

[jhpalmieri]

comment:33

Replying to Dima Pasechnik:

Replying to Matthias Köppe:

I think it makes sense to downgrade the gcc package to "experimental", which would reflect the fact that we no longer test gcc_spkg configurations on our CI.

downgrading to experimental means inevitable bitrot - happening already, right? If noone uses it in practice, I see no point in keeping it, however quick it can be built.

Our experimental packages are a mess in general, so our gcc package would fit right in, the more bitrot the better. I also don't object to removing it altogether — I don't really care one way or the other. But I think we should keep gfortran since it builds on a range of platforms and serves a purpose, at least on OS X.

[dimpase]

comment:34

removing gcc package, while retaining gfortran package, would be a step in right direction. Would we like to proceed with this?

It'd certainly simplify the convoluted build logic of potentially having two C compilers around, and lead to less reports of build failures.

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,23 +1,9 @@
-This is a followup to #32060. Building gcc from source is very hard, and unnecessary for any platform Sage supports now. 
-
-Same applies to gfortran, with macOS being an exception - there is no (g)fortran coming from the system.
-
-So we need such an easy to install replacement, e.g. one provided by Conda, Homebrew, MacPorts, etc.
-
-Should the need arise for something more isolated, (hopefully not), there are experimental standalone distributions of gfortran for macOS, by the gfortran maintainer of the Homebrew package. This offers a way to entirety remove gcc and gfortran spkgs from Sage. 
-
-​https://github.com/fxcoudert/gfortran-for-macOS - specifically, 
-https://github.com/fxcoudert/gfortran-for-macOS/releases/tag/11.2-bigsur-intel
-provides a signed and notarized installed of gcc 11.1 to `/usr/local` (I tested it, it works)
-
-
-Other Fortran possibilities are various closed source offers, e.g. from Intel and NAG,
-as well as open-source flang, etc.
-
 Discussions:

 - https://groups.google.com/g/sage-devel/c/cSsAsPuVnxg/m/yVSGUFi7BQAJ (June 2021)
 - https://groups.google.com/g/sage-devel/c/NfUKjAaTcUg/m/JwqDd7mVAwAJ (September 2021)

+In this ticket:

+- Just downgrade the gcc spkg to experimental, to match our recommendations in the installation guide in #34528.

[mkoeppe]

Dependencies: #34528

[dimpase]

Branch: u/dimpase/packages/gcc_experimental

[dimpase]

Changed dependencies from #34528 to none

[dimpase]

New commits:

58ca5d2

gcc->experimental, gfortran->optional

[dimpase]

Commit: 58ca5d2

[dimpase]

Author: Dima Pasechnik

[mkoeppe]

comment:40

Replying to Matthias Köppe:

I think it makes sense to downgrade the gcc package to "experimental"

I withdraw this suggestion. Since gcc is a prerequisite of standard packages, it cannot be anything but standard. Likewise for gfortran.

[jhpalmieri]

comment:42

The question is whether it should be viewed as a prerequisite, something that Sage should not at all be responsible for installing. gfortran is not directly available on OS X, so I can see why we would keep that, but gcc is not so clear. I am not up to date on linux distributions; are there distributions that we support that do not provide a sufficiently current gcc?

[jhpalmieri]

comment:43

Also, I think our gcc package currently fails to build to M1/M2. Is the work involved to fix it worth the effort?

[mkoeppe]

comment:44

Replying to John Palmieri:

I am not up to date on linux distributions; are there distributions that we support that do not provide a sufficiently current gcc?

I think by definition we only support distributions that provide a suitable gcc.

But keeping gcc costs us nothing, given that its build script is shared with gfortran and we promise nothing about it other than providing it as a last resort for users as described https://trac.sagemath.org/wiki/ReleaseTours/sage-9.8#SupportforGCC8droppedC17featuresnowavailable.