Open dimpase opened 7 months ago
As author of the original mission statement, I do agree that much has changed in the world in the last 20 years, as outlined here.
a. Sage should be a good, economic citizen of Python universe; it should not duplicate efforts elsewhere.
This is an important point. To make it more concrete, I would add that existing solutions should be reused and existing standards should be followed whenever possible. Sage currently has many complex 'customizations', that might have been improvements over the state-of-the-art solutions at the time they were introduced, but are now preventing an easy adaptation of the new standards (examples are custom package manager, custom test runner, custom doctest format, custom sphinx setup).
I also think that the mission statement is no longer appropriate.
But the perhaps most important point is missing in the list above:
This also means that focusing the discussion on the Sage distribution in its role of providing the Sage "application" to end users is fundamentally misguided. (See the Apr 2023 sage-devel thread "What was/is/will be the purpose of maintaining the Sage distribution?"
Instead, we need to focus on what the Sage distribution provides for the development and distribution activities of Sage (as an upstream project).
Dima and Matthias: do I have your permissions to delete all the off topic argumentative comments above? Please just answer yes or thumbs up. Thanks!
[Edit: I've deleted most of this discussion. Please have a civil and useful discussion. Thanks!]
On 6 December 2023 19:36:24 GMT, William Stein @.***> wrote:
Dima and Matthias: do I have your permissions to delete all the off topic argumentative comments above? Please just answer yes or thumbs up. Thanks!
sure
It's hard to come up with catchy short mission statements, but let me propose one:
Enable and utilize open-source mathematical libraries within Python realm
Comments welcome. Please feel free to spread the word.
@kcrisman @vbraun @videlec @nthiery @fredrik-johansson @isuruf @fchapoton @jhpalmieri @VivianePons @tornaria @tscrim @fperez
This also means that focusing the discussion on the Sage distribution in its role of providing the Sage "application" to end users is fundamentally misguided.
So whose role is this? For those who have no idea what wheels and reference distributions are? In the thread mentioned, "We decided to get out of the business to publish binaries; we weren't doing it well, and numerous distributions and third-party binaries filled the role." The first part may be true, but I am not too convinced by the latter for Windows, and even for Mac we do rely on the kindness of the 3-manifolds project.
I'm trying not to be too snarky on this, but Sage is still a tool, and if the mission statement changes, at least I personally would like it to be a tool that is available to the unwashed masses. And that means, now more than ever, "apps" that can be downloaded and installed very easily. (As an example, the demise of the Android app using the cell server, while understandable, was a pity.) One doesn't have to invoke Maple or Mma to say that providing the Sage "application" to end users is a really valuable thing for someone to do. And who else would?
Maybe Docker images are part of this, but that's a pretty weighty solution. Maybe conda is, but I haven't seen consensus on that yet. Maybe CoCalc is part of this, but probably not for everyone, not least because not everyone in the world has 100% connectivity at all times, nor may be allowed to use cloud solutions (for various reasons). Likewise the cell server, though it is fantastic for many things, and probably has more Sage users than all Linux ones combined. If we don't provide a GUI, then how does a user install it? Do they know? Do they need to delve into CLI to do it? If someone is told to download Jupyterlab and then get a Sage kernel, is this something someone can do who is just starting a programming course and has never seen a command line before? Maybe no one of these questions is important on its own, but together they are crucial.
@williamstein 's most brilliant idea (and no disrespect intended, or surely taken, with respect to his other brilliant ideas!) was to be open to it when @wdjoyner asked him if Sage could do calculus. And once Sage went beyond just providing a few localized research tools, but a full ecosystem to take the young mathematician from day one of college to the frontiers of research, the role of end-user distribution needs to be part of the discussion. Otherwise I don't really see the point of having a mission statement, or Sage qua Sage, at all.
The alternative would be dozens of independent Python modules which may or may not interface with various C libraries. In the end, that would be a real loss to the open-source mathematics community. Sage's strength is precisely in allowing people to seamlessly use calculus, graphs, representations, statistics, number fields, and a bunch of other things all in one computation (including from dependencies like GAP, Pari, R, etc.) using a friendly and well-supported base language.
So as not to be charged with only complaining, which could be justified after that long of a comment:
Enable and utilize open-source mathematical libraries within Python realm
This is hardly objectionable as a goal for something, but Sage is really more than the union of its parts. How about this? Surely too long, but brevity isn't my strong suit, as amply demonstrated here.
Unify and interweave open-source mathematical libraries, using the full Python ecosystem as a framework
I think that the mission statement should address the needs and interests of users, not developers, packagers, distributors, etc.
The idea that Sage aims to make existing open source mathematical software available to its users in a consistent convenient computing environment should be the main component of the mission statement.
Developing and expanding the Sage environment should be the primary goal of Sage developers. Other things, even core features such as the choice of using Python as the framework in which to implement the environment, are implementation details. The reason for using Python should be that the Sage developers found it to be the best choice for how to implement the Sage computing environment.
I also have a suggestion for a secondary component of the mission statement. This does not apply directly to Sage users. Instead it applies to the community of which Sage is a part. And it also relates to modularization.
I think Sage should aim to support other open source computing environments by enabling them to conveniently embed parts of Sage.
Modern mathematics with an integration mission
Is that a self-referential mission statement? 😊
The most important property of a mission statement for SageMath is that it suggests answers to questions about the direction and scope of the project, and also helps people who find out about Sage to know what Sage is. The current mission statement has been very helpful in that regard many times over the years. E.g., if somebody said "Maple is awesome because it can do XX, and we really need that functionality to be able to use Sage instead", the mission statement would suggest that adding XX to Sage would be reasonable (for many values of XX). If somebody said "Sage should be rewritten in Julia", the mission statement would suggest: "nope, since that doesn't clearly help people who need a viable alternative to the Ma's, since they mostly don't care about the underlying implementation language.". If the mission statement were instead about increasing the use of Julia in the world, then such a suggestion would be strongly supported by the mission statement (obviously).
A shortcoming of the current mission statement is that it suggests a broader scope of functionality than the core Sage library actually has, e.g., Matlab involves a ton of functionality (simulink) that has nothing to do with the Sage library, at least.
I agree that the original mission statement has served us greatly and is still good.
On the other hand, Sage has grown so big that the competitive and inferiority complex mindset against Magma, Maple, Mathematica and Matlab is now inappropriate. In revising the Sage documentation, I removed a few sentences from such sentiments.
So I suggest (we are) "Creating open source mathematics software in Python ecosystem".
Maybe Python is an implementation detail, but it is also true that Python is the primary reason that Sage exists.
Keywords I care about:
So that could make something like:
"Sharing, integrating, and leveraging libre computational mathematics"
or:
"libre integrated computational mathematics at the fingertips of all"
I think that the mission statement should address the needs and interests of users, not developers, packagers, distributors, etc.
I would go even further: it shouldn't just addres users or potential users. It should find anchors in the mind of every reader, mathematician or not. It is more important that it sticks in the memory of those who then stop reading, rather than those who continue reading. The best way to achieve this is to use metaphors, for example:
.. be the Organ in the Cathedral of Mathematics with registers the Open Source Computer Algebra Systems!
I strongly support removing any references to other computer algebra systems from the Sage mission statement.
Concerning the motivation of Dima opening this issue, I may summarize his list by that Python ecosystem has grown much bigger than Sage. I think both Dima and Matthias are making great efforts to reshape Sage to fit better with the changed landscape. All developers would appreciate it. But the devil is in details. Whatever changed mission statement we agree upon won't dictate what we decide on details. The mission statement is just our north star. Sage has the inertia from the 20 years legacy, and won't change the direction in a day. I hope that the leading developers are not in hurry and adjust the wheel little by little in the given situation.
The best way to achieve this is to use metaphors
it's hard to come with culturally neutral metaphors, e.g. I'd avoid any cathedrals etc.
Since it's very hard to come up with a short slogan that captures also all these different aspects, I suggest to instead write a short "mission statement" that encompasses each point with a short explanation. In form (but obviously not in content), something similar to https://openai.com/charter. The current "Creating a viable free open source alternative to Magma, Maple, Mathematica and Matlab" could still serve as a good advertisement slogan (it's catchy, conveys quite well the general scope and people find sage by googling "open source alternative to Ma*").
Here is a concrete suggestion for a Sage mission statement:
The mission of Sage is: first, to provide convenient access to high quality mathematical software within a single uniform Python-based computing environment; second, to support high quality open source software projects by embedding such projects into Sage and by making it possible for components of Sage to be embedded in other projects.
Mission statements that concentrate primarily on the packaging miss out on something the current mission statement does cover and that actually happens: providing new functionality. A slogan used to be "don't reinvent the wheel but build the car". It means packaging/using existing software is a means to an end, not the primary goal. The goal is the functionality. If the functionality isn't available elsewhere, or not in a satisfying state, then developing something new is entirely reasonable.
There are various examples of new functionality primarily implemented within sage, making use of the infrastructure and libraries that sage provides. In the long run, there is value in having such functionality integrated in sage itself and not just as external packages, where they will bitrot and eventually not work with more modern sage versions.
I don't immediately have a good proposal for a mission statement, but I think we do want a new mission statement to be inclusive of functionality developed primarily within sage; not just packaging and gluing of functionality provided by other packages.
I had in mind that those innovations would be included as part of the "single uniform Python-based computing environment". Maybe it could somehow be emphasized that the environment has important valuable content of its own, beyond the open source projects that are embedded in it.
So I suggest (we are) "Creating open source mathematics software in Python ecosystem".
Complementing with some catchy adjectives,
(we are) "Creating open source mathematics software in Python ecosystem, comprehensive, modern, and cutting edge".
Bard: read https://github.com/sagemath/sage/issues/36827 and write a better mission statement
To empower the world of mathematics with a unified, open-source ecosystem that harnesses the power of Python and enables seamless collaboration and sharing of mathematical knowledge.
ChatGPT: copy & paste content, then:
Empowering collaborative mathematics in the Python ecosystem through the integration, development, and utilization of open-source mathematical libraries.
Of course the chatbots love phrases like "Python ecosystem" since they have no meaning. "The better to confuse you with, my dear". When the word "ecosystem" was coined by Arthur Roy Clapham in 1935 it was also given a definition by Arthur Tansley, the ecologist. But I have no idea how to decide whether some piece of software is in the "Python ecosystem", nor whether it might be a predator or a parasite within that ecosystem. I hope that we can emerge from this exercise with a mission statement that does not use that phrase.
Also, I don't really think that collaboration is part of the current Sage mission, as it is currently applied by the Sage developers. CoCalc is a completely different story, of course.
I have no idea how to decide whether some piece of software is in the "Python ecosystem"
At this point in time, the phrase "python ecosystem" is very commonly used and defined on a large number of websites. Browse [1].
A good test -- "is it easy to install from PyPi?" Or, as [2] says "At the heart of the Python ecosystem is the Python Package Index, commonly known as PyPI." By this definition, libraries such as scipy, PyTorch and Tensorflow and networkx are clearly in the python ecosystem, and SageMath is NOT in the Python ecosystem. Also, if you look at sites linked to from [1] you'll see SageMath is notably not listed. Note that Pypi is key here, not conda, as conda is a different ecosystem.
I personally think by far the single most important thing the SageMath project could do during the next 2 years would be to join the Python ecosystem, i.e., be "easily installable via pip from Pypi". A first step is for SageMath devs to realize that "the Python ecosystem" is a thing, perhaps one of the most important and interesting things in software today.
[1] https://www.google.com/search?q=what+is+the+python+ecosystem [2] https://medium.com/@instailyacademy/understanding-the-python-ecosystem-a-comprehensive-guide-1fe78b7c6440
I have no idea how to decide whether some piece of software is in the "Python ecosystem", nor whether it might be a predator or a parasite within that ecosystem.
Indeed :)
Also, I don't really think that collaboration is part of the current Sage mission
Nowadays there are many ways to do collaborative coding in Python and other systems, apart from CoCalc (gitpod, GitHub's codespaces, etc), so this need not be a part of Sage's mission anyway.
A good test -- "is it easy to install from PyPi?" Or, as [2] says "At the heart of the Python ecosystem is the Python Package Index, commonly known as PyPI." By this definition, libraries such as scipy, PyTorch and Tensorflow and networkx are clearly in the python ecosystem, and SageMath is NOT in the Python ecosystem.
I would say that a pypi package which is only available as an sdist, not as a binary wheel, is NOT easy to install.
I personally think by far the single most important thing the SageMath project could do during the next 2 years would be to join the Python ecosystem, i.e., be "easily installable via pip from Pypi".
Thus my interpretation of this would be that there should be a binary wheel for each component of Sage. This is definitely possible, and I am in favor of doing it. But I note that this approach is antithetical to the current sage build philosophy, which says that Sage should attempt to use shared libraries installed on the host system whenever possible. Binary wheels must include all libraries needed by the Python extension. So incorporating this new philosophy into the Sage mission would also imply making a huge change in the current shared mindset of the Sage developers.
[...] Binary wheels must include all libraries needed by the Python extension. [...]
I don't know the technical details of exactly what's allowed with Python wheels. I've installed a lot of packages from the pytorch/tensorflow deep learning ecosystems lately, and they are (1) definitely part of the Python ecosystem, and (2) depends on major binary components that can't legally be included, e.g., CUDA (or the analogue of MacOS).
I guess I don't really care what the rules are. I just personally wish SageMath was part of the Python ecosystem in the same way that PyTorch/Tensorflow/Scipy etc are.
Thus my interpretation of this would be that there should be a binary wheel for each component of Sage. This is definitely possible, and I am in favor of doing it.
Yes, and we are working on it as part of the modularization project. The first two distribution packages already ship binary wheels (which are built automatically on GH Actions)
Next:
A good test -- "is it easy to install from PyPi?" Or, as [2] says "At the heart of the Python ecosystem is the Python Package Index, commonly known as PyPI." By this definition, libraries such as scipy, PyTorch and Tensorflow and networkx are clearly in the python ecosystem, and SageMath is NOT in the Python ecosystem.
I would say that a pypi package which is only available as an sdist, not as a binary wheel, is NOT easy to install.
If your OS (or pseudo-OS, e.g. Conda or Homebrew) is not developer-hostile, it's usually not a problem.
I personally think by far the single most important thing the SageMath project could do during the next 2 years would be to join the Python ecosystem, i.e., be "easily installable via pip from Pypi".
Thus my interpretation of this would be that there should be a binary wheel for each component of Sage. This is definitely possible, and I am in favor of doing it. But I note that this approach is antithetical to the current sage build philosophy, which says that Sage should attempt to use shared libraries installed on the host system whenever possible. Binary wheels must include all libraries needed by the Python extension. So incorporating this new philosophy into the Sage mission would also imply making a huge change in the current shared mindset of the Sage developers.
auditwheel and its analogs on other OSs can add a correct dynamic library to a wheel. So it's not hard to imagine a similar tool (don't know if it exists or not) replacing links to a bundled dynamic library with links to an ABI-compatible library elsewhere.
If your OS (or pseudo-OS, e.g. Conda or Homebrew) is not developer-hostile, it's usually not a problem.
It is not a problem for developers. For users it is a major problem. And this is exactly what I was talking about when I said at the top of this thread:
I think that the mission statement should address the needs and interests of users, not developers, packagers, distributors, etc.
Thus my interpretation of this would be that there should be a binary wheel for each component of Sage. This is definitely possible, and I am in favor of doing it.
Yes, and we are working on it as part of the modularization project. The first two distribution packages already ship binary wheels (which are built automatically on GH Actions)
Bravo!
Please forgive me if I add that the same has been true for cypari for years -- it even has binary wheels for Windows. Unfortunately, that was the one aspect of cypari that was not adopted (or improved upon) by cypari2.
the same has been true for cypari for years -- it even has binary wheels for Windows.
I know! My hope is that the modularization also enables porting efforts of other parts of Sage to native Windows (and other platforms).
Unfortunately, that was the one aspect of cypari that was not adopted (or improved upon) by cypari2.
Indeed. And bringing this to cypari2 (or even finding a way to merge the two projects again, as we discussed a while ago) will have a high priority.
That's one of the items in:
I don't know the technical details of exactly what's allowed with Python wheels. I've installed a lot of packages from the pytorch/tensorflow deep learning ecosystems lately, and they are (1) definitely part of the Python ecosystem, and (2) depends on major binary components that can't legally be included, e.g., CUDA (or the analogue of MacOS).
According to scipy (who misuses the verb "to vendor", according to the definition that I only recently learned):
When python -m build or pip wheel is used to build a SciPy wheel, that wheel will rely on external shared libraries (at least for BLAS/LAPACK and a Fortran compiler runtime library, perhaps other libraries). Such wheels therefore will only run on the system on which they are built. See the pypackaging-native content under “Building and installing or uploading artifacts” for more context on that.
A wheel like that is therefore an intermediate stage to producing a binary that can be distributed. That final binary may be a wheel - in that case, run auditwheel (Linux), delocate (macOS) or delvewheel (Windows) to vendor the required shared libraries into the wheel.
If your OS (or pseudo-OS, e.g. Conda or Homebrew) is not developer-hostile, it's usually not a problem.
It is not a problem for developers. For users it is a major problem. And this is exactly what I was talking about when I said at the top of this thread:
I think that the mission statement should address the needs and interests of users, not developers, packagers, distributors, etc.
Under a "user" I understand someone who is able to open a terminal and run there a couple of things like sudo apt-get install foo bar && make baz
, with at least a vague understanding of what this is meant to be doing.
I don't expect a user to be able to do much more than that, but teaching the difference between Python and shell prompts is not a part of Sage's mission.
Under a "user" I understand someone who is able to open a terminal and run there a couple of things like
sudo apt-get
[...]
Some people might argue that seems more like an "admin" than a user.
Of course the chatbots love phrases like "Python ecosystem" since they have no meaning. ...
I am not a chatbot (but I don't know how I can prove this :-)
The term is ambiguous as much as the term ecosystem. Certainly we are making efforts to connect Sage with the Python community and networks. Hence the term deserves to be in the mission statement. It is not necessary that Sage is now definitely within the Python ecosystem. Sage has never been a perfect alternative to 4 MAs.
Of course the chatbots love phrases like "Python ecosystem" since they have no meaning. ...
I am not a chatbot (but I don't know how I can prove this :-)
The term is ambiguous as much as the term ecosystem. Certainly we are making efforts to connect Sage with the Python community and networks. Hence the term deserves to be in the mission statement. It is not necessary that Sage is now definitely within the Python ecosystem. Sage has never been a perfect alternative to 4 MAs.
When thinking about good mission statement wording, we should keep in mind that these lines provide the first impression of a potential Sage newcomer when trying to figure out what Sage is. For these people it must also serve as a kind of advertising banner. This means that it must be easy to understand even for people without Python experience (here I assume and hope that we welcome them too). From this perspective, phrases like “Python ecosystem” should not be used.
On the other hand, I agree that the North Star of our current development should be the Python ecosystem. But I think this is more of a goal than a mission.
I like @culler 's suggestion (https://github.com/sagemath/sage/issues/36827#issuecomment-1848606347), but I think it needs some improvements to address @nbruin 's criticism more explicitly.
I think we would do well to remember that Sage meant to be a software package and not a collection of libraries run from Python. For example, I don't want to type 1/2
and get 0.5
, nor do I want to have to import lots of things every time I start things up. (Of course, there can be scripts, but this means I cannot go from computer to computer and it is a more advanced thing compared to a casual user.)
To continue with the build-the-car metaphor, it seems like the way we are trying to break things up to fit into the Python ecosystem is akin to Sage is now just bigger components to the car, but some assembly is still required. Of course, it would be good to be installable via PyPI, but I think Sage should have an all-inclusive installation along with an easy method to have an isolated installation for very casual users.
Subsequently, I would firmly reject any mission statement that mentions being a part of the Python ecosystem. I think we should have our mission statement be something like:
"Free open source mathematical software for teaching, research, and the world"
or slightly less grandiose at the end.
I fully agree. If mentioning "Python ecosystem" is understood to mean that we aim for breaking Sage into pieces, I would reject that too :-) Thus my second amendment to my own proposal,
(we are) "Creating open source mathematics software on Python, comprehensive, modern, and cutting edge".
Here is my second attempt:
The mission of Sage is to produce a comprehensive interactive mathematical computing system which is freely and conveniently available to all mathematicians, from students to educators to researchers. Sage incorporates many other high quality open source mathematical software projects, making them all accessible within a single, uniform, computing environment. Sage supports the development of open source mathematical software by distributing components of Sage through the standard Python distribution channels and by designing them to be embeddable into other software projects.
I've been lurking and thinking on this issue, and I couldn't come up with something good to say until I read the proposal by @tscrim which I liked the most by far.
I'll repeat it here, slightly edited to my taste:
Free open source mathematical software for research and teaching
It's short and catchy, and reflects precisely what I would like sagemath to be. I took out the "and the world" part but maybe that or something similar is still needed, not sure.
IMO sagemath should also re-adopt the "don't reinvent the wheel but build the car" slogan which was such a powerful motor for SAGE in the early years.
Il semble que la perfection soit atteinte non quand il n’y a plus rien à ajouter, mais quand il n’y a plus rien à retrancher.
I don't understand why "python ecosystem" should be the focus of sage distribution. And even less understand why it should be part of the mission statement. Python ecosystem rather sounds as an implementation detail of a strategy to make sage widely available.
To come back to the first point, sage distribution preference order should roughly be
It would be wonderful to have sage on pip and a reasonable goal to achieve in the near future. But I don't think that it deserves to be the "mission".
I'll repeat it here, slightly edited to my taste:
Free open source mathematical software for research and teaching
Here I'm missing an ourstanding feature of Sage. The slogan might be true for many other open source mathematical software, too. At least we should add and exploring mathematics which will be more difficult to do self-educated with other systems.
For me the most important outstanding feature is: Sage incorporates many other high quality open source mathematical software projects, making them all accessible within a single, uniform, computing environment (cited from https://github.com/sagemath/sage/issues/36827#issuecomment-1852100385).
This means you don't have to learn the individual user interfaces of GAP, Singular, or whatever to use their functionality. You may not even be aware of them being used because some experts have decided which third-party tool is best to solve the specific problem. I think this is the feature that makes Sage comparable to the 4 MA's: All in one!
Also worth highlighting are the numerous, permanently tested and very useful examples of all functionality in the reference manual!
IMO sagemath should also re-adopt the "don't reinvent the wheel but build the car" slogan which was such a powerful motor for SAGE in the early years.
+1
Free open source mathematical software for exploring mathematics
This makes me recall something I saw in a little piece of paper a long time ago @williamstein
S ystem for
A rithmetic
G eometry
E xperimentation
I don't understand why "python ecosystem" should be the focus of sage distribution. And even less understand why it should be part of the mission statement. Python ecosystem rather sounds as an implementation detail of a strategy to make sage widely available.
To come back to the first point, sage distribution preference order should roughly be
- source
- (linux) distribution
- conda
- pip
It would be wonderful to have sage on pip and a reasonable goal to achieve in the near future. But I don't think that it deserves to be the "mission".
except that for a typical user source
comes last, not first.
Ecosystem is not an implementation detail, it's a very important feature. If Sage does everything its own often suboptimal way, users would not come, and we actually see this. Sage is an outlier, one has to walk a long way to comfortably work with Sage.
The original Sage mission statement ("Creating a viable free open source alternative to Magma, Maple, Mathematica and Matlab") appeared close to 20 years ago, and since then there were big changes in Python and non-Python maths computing landscape, as well as in Sage:
Nevertheless, one sees the (old) mission statement invoked now and then to defend the current uncontrolled growth of the number of packages, continued vendoring of most everything, etc. Therefore the need to amend the mission statement is crucial, we don't want it to lead to complete unraveling of the project.
While I can't at the moment of writing this produce a short new version, here are few points which I see as crucial
a. Sage should be a good, economic citizen of Python universe; it should not duplicate efforts elsewhere. Existing solutions should be reused and existing standards should be followed whenever possible. Sage currently has many complex 'customizations', that might have been improvements over the state-of-the-art solutions at the time they were introduced, but are now preventing an easy adaptation of the new standards (examples are custom package manager, custom test runner, custom doctest format, custom sphinx setup).
b. Sage (the distribution) should slim down, in view of a; e.g. there is no need to ship its own GUI or a set of compilers.