Installation manual: Add decision tree, remove mention of Sage-mirror-hosted binary distributions

[mkoeppe]

The binary distributions on the Sage mirrors are unmaintained (#31133) and have been broken for a long time.

We update the manual by including a decision tree. It never leads to the recommendation to use these binary distributions because better alternatives are available for every use case.

Hence, we replace all details regarding binaries by a short notice that they have been discontinued.

PR to also remove mention of the binaries from the website:

• https://github.com/sagemath/website/pull/228 (merged)

CC: @dimpase @jhpalmieri @vbraun @collares @isuruf

Component: distribution

Author: Matthias Koeppe

Branch/Commit: 0f6ade5

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/33131

[mkoeppe]

Description changed:

--- 
+++ 
@@ -3,5 +3,6 @@
 We direct people instead to
 - conda-forge
 - nix
+- docker
 - build from source.

[mkoeppe]

Branch: u/mkoeppe/installation_manual__remove_mention_of_binary_distributions

[mkoeppe]

Author: Matthias Koeppe

[mkoeppe]

Commit: 480ec86

[mkoeppe]

New commits:

480ec86

src/doc/en/installation: Remove mention of binaries

[mkoeppe]

Description changed:

--- 
+++ 
@@ -6,3 +6,5 @@
 - docker
 - build from source.

+PR to also remove mention of the binaries from the website: https://github.com/sagemath/website/pull/228
+

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,4 +1,4 @@
-The binary distributions on the Sage mirrors are unmaintained and have been broken for a long time.
+The binary distributions on the Sage mirrors are unmaintained (#31133) and have been broken for a long time.

 We direct people instead to
 - conda-forge

[orlitzky]

comment:9

Huge +1 from me. If you want a binary, you should be getting it with (say) apt-get.

[dimpase]

comment:10

yes! +100

[7822f248-ba56-45f1-ab3d-4de7482bdf9f]

comment:11

Binary distributions are a "necessary evil" for platforms where building Sage can be considered as needing "system fu" out of proportion of "normal" using Sage fu.

Cases in point :

• Windows

Notwithstanding E. Madison Bray and others' gigantic efforts, building the Cygwin port of Sage is difficult ; probably too difficult for casual Windows users. The problem is that this puts Sage above the reach of at least 90% of undergraduate science/math students, i. e. our core target audience for future users.

The future of Sage on Windows may well reside in WSL2. However, this solution implies the care and feeding of a Linux virtual machine on Windows. Easier, but again out of the reach of at least 50% of our target audience (Unif fu isn'a as popular among science/engineering students ast it once was, don't get me started on biology/geology students, and don't even think of physicians-to-be...:-)).

• Mac

Not approaching a Mac at least than ten yards, I can't possibly knowledgeably comment on this platform ; I am still impressed by the sheer volume of Mac-specific or Mac-related problems that flourish on sage-release at each release...

Again, this platform represents a non-negligible part of our target audience.

These two platforms are a special case, requiring IMHO special consideration...

[7822f248-ba56-45f1-ab3d-4de7482bdf9f]

comment:12

Replying to @orlitzky:

Huge +1 from me. If you want a binary, you should be getting it with (say) apt-get.

Agreed. For Linux (Unix) users.

Which are nowadays a dwindling minority (if an elite one, of course ;-)). See below my remarks on Windows and Macs...

And, BTW, a Debian package is a binary distribution...

[dimpase]

comment:13

Replying to @EmmanuelCharpentier:

Binary distributions are a "necessary evil" for platforms where building Sage can be considered as needing "system fu" out of proportion of "normal" using Sage fu.

Cases in point :

• Windows

use Docker.

• Mac

Not approaching a Mac at least than ten yards, I can't possibly knowledgeably comment on this platform ; I am still impressed by the sheer volume of Mac-specific or Mac-related problems that flourish on sage-release at each release...

Again, this platform represents a non-negligible part of our target audience.

for Mac there is a binary installer provided by Snappy group.

[pipedream]

comment:14

The ticket is to remove it from the installation documentation. I'm absolutely fine with that if it improves the situation for developers and users.

I would still like access to easily download binary distributions.

I recently on Linux (Debian) had problems building a newer version than apt but th ebinary distribution worked fine. I also do have problems with the numpy SIGILL on older CPUs (which the Debian versions don't have; don't know why).

I'd love to have Debian backports and sloppy-backports or third party (preferably sagemath not launchpad) repositories with latest sagemath packages for Debian releases and Ubuntu LTSes at least. Might be a tall order for volunteers to maintain.

[orlitzky]

comment:15

Replying to @pipedream:

The ticket is to remove it from the installation documentation. I'm absolutely fine with that if it improves the situation for developers and users.

I would still like access to easily download binary distributions.

That's not an option either way:

• https://www.sagemath.org/download-windows.html
• https://www.sagemath.org/download-linux.html
• https://www.sagemath.org/download-mac.html

There are many unfixed problems -- some fundamental -- and to put it bluntly, nobody cares. This ticket just changes the documentation to reflect reality, and perhaps in the future will allow us to not be held back by "what about the binaries?!" arguments.

[edd8e884-f507-429a-b577-5d554626c0fe]

comment:16

I think the question is not whether the binaries are maintained or not on our side but whether they are useful to users. I used to build up to 16 binaries at each release some years ago, then stopped since Jan Groenwald at AIMS dedicated some machines for that. Maybe he stopped after many years, but it is not an individual question. If those binaries are useful, then we should put some energy to dedicate some infrastructure for them, not the other way. I have the impression that the same happened with gitlab workers (they were usefull but since they were unmaintained, we stopped them).

Regarding MacOS binaries, why not shipping the ones produced by 3-manifolds project on the Sage mirrors ?

[orlitzky]

comment:17

Replying to @sagetrac-tmonteil:

I think the question is not whether the binaries are maintained or not on our side but whether they are useful to users.

It would be useful to our users if they could hop on a zoom call and have a sage developer personally install sage for them. But is that a good use of our time? Is anyone actually going to volunteer to do it?

I would say it's the other way around: if no one is willing to fix all of the bugs and commit to producing and supporting the sage binary releases, then the question of whether or not it would be useful is irrelevant. The source (or distro-based) installations, on the other hand, are actively developed and supported.

[mkoeppe]

comment:18

Replying to @sagetrac-tmonteil:

Regarding MacOS binaries, why not shipping the ones produced by 3-manifolds project on the Sage mirrors ?

The maintainer prefers to release them on GitHub releases.

[mkoeppe]

comment:19

Replying to @EmmanuelCharpentier:

• Windows

This ticket makes no changes regarding this platform.

[mkoeppe]

Description changed:

--- 
+++ 
@@ -6,5 +6,6 @@
 - docker
 - build from source.

-PR to also remove mention of the binaries from the website: https://github.com/sagemath/website/pull/228
+PR to also remove mention of the binaries from the website: 
+- https://github.com/sagemath/website/pull/228 (merged)

[williamstein]

comment:22

"We direct people instead to: conda-forge, nix, docker, build from source."

It would be nice to include all the known ways to run SageMath online as well, e.g., Sage cell server (https://sagecell.sagemath.org/), CoCalc (https://cocalc.com), etc. Actually, I don't know of any others, but maybe somebody could make a mybinder (https://mybinder.org/) of Sage, if it doesn't exceed the size limits.

[slel]

comment:23

Replying to @williamstein:

It would be nice to include all the known ways to run SageMath online as well, e.g., Sage cell server (https://sagecell.sagemath.org/), CoCalc (https://cocalc.com), etc. Actually, I don't know of any others, but maybe somebody could make a mybinder (https://mybinder.org/) of Sage, if it doesn't exceed the size limits.

I once made a list but it was missing actual Binder links:

• answer to Ask Sage question 42876

OpenDreamKit's "try" page

• https://opendreamkit.org/try/

and the "More SageMath tutorials" page

• https://more-sagemath-tutorials.readthedocs.io/

both use the same Binder link, which currently does not work.

Might not be that hard to fix...

[vbraun]

Replying to @mkoeppe:

We direct people instead to

• conda-forge
• nix
• docker

That would be nice but the proposed patch mentions neither, nor an distro-specific installation. There are surely issues with the binary packages but they can work for some whereas the current patch isn't helpful at all.

At the very least we should have some eli5 instructions for how to get conda up and running and then install the conda sagemath package. Since conda is really the only way that works without root permissions I'd put that front and center.

[tobiasdiez]

comment:26

Replying to @EmmanuelCharpentier:

The future of Sage on Windows may well reside in WSL2. However, this solution implies the care and feeding of a Linux virtual machine on Windows. Easier, but again out of the reach of at least 50% of our target audience (Unif fu isn'a as popular among science/engineering students ast it once was, don't get me started on biology/geology students, and don't even think of physicians-to-be...:-)).

Setting up sage with WSL should be pretty straightforward. First install WSL with Ubuntu using the Windows store (at least on Win 11, https://aka.ms/wslstorepage, otherwise can be done using a few cmds) and then use apt get to install sage in it.

---

+1 for removing binary distributions for linux, cannot say anything about mac.

---

The information for how to install sage is pretty scattered (wiki, website, docs, readme, ...). I think this should be combined in one place, listing the preferred installation method for each system and describe alternative installation methods as subpages.

[dimpase]

comment:27

our mac binaries are really broken - and there is an alternative.

[mkoeppe]

comment:28

Replying to @vbraun:

Replying to @mkoeppe:

We direct people instead to

• conda-forge
• nix
• docker

That would be nice but the proposed patch mentions neither

... because all of these are already in the manual.

[edd8e884-f507-429a-b577-5d554626c0fe]

comment:29

Replying to @mkoeppe:

Replying to @sagetrac-tmonteil:

Regarding MacOS binaries, why not shipping the ones produced by 3-manifolds project on the Sage mirrors ?

The maintainer prefers to release them on GitHub releases.

If our binaries are broken and those are fine, why not just use their build scripts ? Otherwise, why not ship those working binaries on the Sage mirrors ?

[edd8e884-f507-429a-b577-5d554626c0fe]

comment:30

Replying to @tobiasdiez:

Replying to @EmmanuelCharpentier:

The future of Sage on Windows may well reside in WSL2. However, this solution implies the care and feeding of a Linux virtual machine on Windows. Easier, but again out of the reach of at least 50% of our target audience (Unif fu isn'a as popular among science/engineering students ast it once was, don't get me started on biology/geology students, and don't even think of physicians-to-be...:-)).

Setting up sage with WSL should be pretty straightforward. First install WSL with Ubuntu using the Windows store (at least on Win 11, https://aka.ms/wslstorepage, otherwise can be done using a few cmds) and then use apt get to install sage in it.

I had a look some times ago, but i could not find anything (prehaps because i am not user to windows) : it is possible to produce an single installer (.exe), that does the job of activating WSL, installing a GNU/Linux distro, install Sage in it, and add a launcher on the desktop ?

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

b78260b	src/doc/en/installation: Remove mention of binaries
93df828	src/doc/en/installation/index.rst: Replace section by a decision tree

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 480ec86 to 93df828

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

5af3d04

src/doc/en/installation/index.rst: Add header for toc

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 93df828 to 5af3d04

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

76d2d2a

src/doc/en/installation/index.rst: Typos

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 5af3d04 to 76d2d2a

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,10 +1,8 @@
 The binary distributions on the Sage mirrors are unmaintained (#31133) and have been broken for a long time.

-We direct people instead to
-- conda-forge
-- nix
-- docker
-- build from source.
+We update the manual by including a decision tree. It never leads to the recommendation to use these binary distributions because better alternatives are available. 
+
+Hence, we replace all details regarding binaries by a short notice that they have been discontinued.

 PR to also remove mention of the binaries from the website: 
 - https://github.com/sagemath/website/pull/228 (merged)

[dimpase]

comment:36

Replying to @sagetrac-tmonteil:

Replying to @mkoeppe:

Replying to @sagetrac-tmonteil:

Regarding MacOS binaries, why not shipping the ones produced by 3-manifolds project on the Sage mirrors ?

The maintainer prefers to release them on GitHub releases.

If our binaries are broken and those are fine, why not just use their build scripts ? Otherwise, why not ship those working binaries on the Sage mirrors ?

ha! it needs all that Apple codesigning setup noone is willing to duplicate, it also is not free...

[mkoeppe]

comment:37

https://wiki.sagemath.org/Distribution has some good information, some of which may not be overly outdated.

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 76d2d2a to c1f4e46

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

c1f4e46

src/doc/en/installation/index.rst: Link also to https://wiki.sagemath.org/Conda

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from c1f4e46 to 0f6ade5

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

0f6ade5

src/doc/en/installation/linux.rst: Link to repology and https://wiki.sagemath.org/Distribution

[mkoeppe]

comment:40

Replying to @williamstein:

"We direct people instead to: conda-forge, nix, docker, build from source."

It would be nice to include all the known ways to run SageMath online as well, e.g., Sage cell server (https://sagecell.sagemath.org/), CoCalc (https://cocalc.com)

These 2 options were already mentioned in the manual, still are after the rewrite done in the ticket

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,6 +1,6 @@
 The binary distributions on the Sage mirrors are unmaintained (#31133) and have been broken for a long time.

-We update the manual by including a decision tree. It never leads to the recommendation to use these binary distributions because better alternatives are available. 
+We update the manual by including a decision tree. It never leads to the recommendation to use these binary distributions because better alternatives are available for every use case.

 Hence, we replace all details regarding binaries by a short notice that they have been discontinued.

[slel]

comment:42

For macOS, I would change:

-      You can choose between two options: A 800 MB disk image
-      with the standard Sage distribution,
-      and a 1 GB disk image that adds many optional packages.
+      You can choose between two options: the standard Sage
+      distribution (800 MB disk image, expands to nearly 3 GB),
+      or an extended distribution with many optional packages
+      (1 GB disk image, expands to nearly 3.5 GB).

[mkoeppe]

comment:43

Please feel free to push improvements to the ticket.

[tobiasdiez]

comment:44

Replying to @sagetrac-tmonteil:

Replying to @tobiasdiez:

Replying to @EmmanuelCharpentier:

The future of Sage on Windows may well reside in WSL2. However, this solution implies the care and feeding of a Linux virtual machine on Windows. Easier, but again out of the reach of at least 50% of our target audience (Unif fu isn'a as popular among science/engineering students ast it once was, don't get me started on biology/geology students, and don't even think of physicians-to-be...:-)).

Setting up sage with WSL should be pretty straightforward. First install WSL with Ubuntu using the Windows store (at least on Win 11, https://aka.ms/wslstorepage, otherwise can be done using a few cmds) and then use apt get to install sage in it.

I had a look some times ago, but i could not find anything (prehaps because i am not user to windows) : it is possible to produce an single installer (.exe), that does the job of activating WSL, installing a GNU/Linux distro, install Sage in it, and add a launcher on the desktop ?

I'm not aware of such a possibility. What one could do is to create a custom WSL image (with sage pre-installed), which people could then use. But in order to distribute this in the store, one would need to contact microsoft: https://docs.microsoft.com/en-us/windows/wsl/build-custom-distro#for-linux-distro-maintainers

Another possibility is to install sage as a graphical app in WSLg (https://github.com/microsoft/wslg). For Win11, this then adds a launcher in the Windows start menu.

[tobiasdiez]

comment:45

Replying to @dimpase:

Replying to @sagetrac-tmonteil:

Replying to @mkoeppe:

Replying to @sagetrac-tmonteil:

Regarding MacOS binaries, why not shipping the ones produced by 3-manifolds project on the Sage mirrors ?

The maintainer prefers to release them on GitHub releases.

If our binaries are broken and those are fine, why not just use their build scripts ? Otherwise, why not ship those working binaries on the Sage mirrors ?

ha! it needs all that Apple codesigning setup noone is willing to duplicate, it also is not free...

If you are a non-profit org, you don't have to pay for the developer account (which is needed for the codesigning): https://developer.apple.com/support/membership-fee-waiver/ Not sure if sage officially falls under this, but maybe worth a try?

[dimpase]

comment:46

Replying to @tobiasdiez:

If our binaries are broken and those are fine, why not just use their build scripts ? Otherwise, why not ship those working binaries on the Sage mirrors ?

ha! it needs all that Apple codesigning setup noone is willing to duplicate, it also is not free...

If you are a non-profit org, you don't have to pay for the developer account (which is needed for the codesigning): https://developer.apple.com/support/membership-fee-waiver/ Not sure if sage officially falls under this, but maybe worth a try?

Unless we have a volunteer who's willing to jump through all the hoops to set this all up (we don't have a proper non-profit set up, despite talking about it for years - that's why in particular our donations (GitHub and "direct") are received via OpenCollective, and not in a more direct and cost-efficient way), it's all idle chatter.

Same for a volunteer to actually get on with the Apple developer cerificates, setting the necessary builds up, etc. (as this is all targeting developing via XCode, it's hit and miss with command line).

[kcrisman]

comment:47

Replying to @orlitzky:

Replying to @pipedream: There are many unfixed problems -- some fundamental -- and to put it bluntly, nobody cares.

As per usual in these arguments, what this means is, "nobody with the expertise to maintain binary distributions cares enough to put in the nontrivial work it demands", which isn't quite the same thing.

Anyway, if someone with the expertise of pipedream aka Jan G. is asking about Linux binaries, then it should go without saying that we need some sustainable, reliable way to provide Mac and Windows binaries (unless we assume everyone is using CoCalc, which I do not think should be the assumption). If Nathan and the Snappy folks are committed to this on Mac and that is easy to advertise everywhere it needs to be, great. From the discussion above, it sounds like the Windows situation is much less stable, and so comment:11 still stands. How easy is the Docker image to use by someone who is expecting a "download, then double-click" interface? If it's no harder to use than our most recent Cygwin-based one and is sustainable to produce, great. And all this has to be easy to find off the main Sage home page. (An advantage to removing from mirrors is people won't get confused by the mirror selection process.)

Unif fu isn'a as popular among science/engineering students ast it once was, don't get me started on biology/geology students, and don't even think of physicians-to-be...:-)

LOL. But interestingly, at UCLA apparently a lot of the latter had used Sage in a lab context (see the book), but I'm not so sure now - see this paper where I only see "Python" not "Sage".

[williamstein]

comment:48

LOL. But interestingly, at UCLA apparently a lot of the latter had used Sage in a lab context (see ​the book), but I'm not so sure now - see ​this paper where I only see "Python" not "Sage".

I spent a day with them once. The first few hits for this search

https://www.youtube.com/results?search_query=william+stein+sagemathcloud+ucla

are to some short videos that came out of that.

The answer is that they use CoCalc (previous called SageMathCloud) with Sage worksheets and Jupyter notebooks, and have done so since 2014. They do not install Sage locally.

[mkoeppe]

comment:49

Replying to @kcrisman:

it sounds like the Windows situation is much less stable, and so comment:11 still stands.

The present ticket improves the Windows situation by giving a clear recommendation by way of the decision tree.

Needs review.