search

sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.47k stars 487 forks source link

Meta-ticket: Modularize sagelib into separate distributions (pip-installable packages) sagemath-... for Sage 10.x #29705

Open mkoeppe opened 4 years ago

mkoeppe commented 4 years ago

We modularize sagelib into separate distributions (pip-installable packages), so that these can be built and run with a much smaller set of dependencies. The packages are made available individually on PyPI in source and wheel form. The modularization is done without changing the overall structure of the source tree by means of PEP 420 implicit namespace packages.

Mega-PR #35095 (not for merging; changes are being cherry-picked from here for integration in Sage) adds the new distributions. Instead of defining a distribution package for every possible community or subfield of mathematics that Sage supports, the modularization project introduces three types of distribution packages:

Distributions named after a basic mathematical structure: The packages may also cover a wide range of generalizations/applications of the structure after which they are named. Users who work in a specialized research area will, of course, recognize what structures they need. The down-to-earth naming also creates discoverability by a broader audience. Not many more distribution packages than these 7 are needed:

Distributions named after a third-party non-Python dependency: This makes technical sense because the dependencies will be localized to this distribution package, but it also helps give attribution and visibility to these libraries and projects that Sage depends on. Standard packages Optional Packages
sagemath-flint sagemath-bliss
sagemath-gap sagemath-brial
sagemath-glpk sagemath-coxeter3
sagemath-giac sagemath-mcqd
sagemath-homfly sagemath-meataxe
sagemath-lcalc sagemath-sirocco
sagemath-libbraiding sagemath-tdlib
sagemath-libecm
sagemath-linbox
sagemath-ntl
sagemath-pari
sagemath-singular

Distributions named after a technical functionality:

Older posts and presentations:

Initial steps: (2020)

  1. In particular, create a package sagemath-objects that makes Sage objects, the element/parent framework, categories, the coercion system and the related metaclasses available. This package would, for example, not provide or know about integers, polynomials, or matrices (thus avoiding all the dependencies on the various libraries implementing these). Ideally it would only have a build dependency on Cython and no runtime dependencies.

    (Doctesting sagemath-objects would still need more of Sage, but some it could also be done by mocking instead of with the real classes.)

    a) These facilities could see a wider use in the mathematical Python community, and in this way we could hope to benefit from a larger developer base. For example, we would hope that sagemath-objects could be ported for use with PyPy instead of CPython.

    b) Building and deploying a user package would be easier if it could depend on a small package such as sagemath-objects only instead of the whole multigigabyteness of Sage.

    This has been completed in Sage 9.6 (#29865).

  2. Remove the mechanism of OptionalExtensions from sagelib.

    Currently, a user would install, for example, the optional package tdlib and then rebuild sagelib so that the OptionalExtension sage.graphs.graph_decompositions.tdlib is built and installed.

    With this ticket, the user would instead install a new optional package sagemath-tdlib (which has tdlib as a dependency); this installs the Extension sage.graphs.graph_decompositions.tdlib (as a namespace package).

    This has been completed in Sage 9.2 (#29701).

Not within the scope of this ticket:

Initially noted constraints and challenges: (2020)

Tickets for Sage 9.2: (2020)

see also Modularization changes in Sage 9.2 (release tour)

Tickets for Sage 9.3: (2021)

see also Modularization changes in Sage 9.3 (release tour)

Tickets for Sage 9.4: (2021)

see also Modularization changes in Sage 9.4 (release tour)

Tickets for Sage 9.5: (2022)

see also Modularization changes in Sage 9.5 (release tour)

Tickets for Sage 9.6: (2022)

see also Modularization changes in Sage 9.6 (release tour)

Tickets for Sage 9.7: (2022)

see also Modularization changes in Sage 9.7 (release tour)

Tickets for Sage 9.8: (2023)

see also Modularization changes in Sage 9.8 (release tour)

Tickets for Sage 10.0 (2023)

Tickets for Sage 10.1 (2023)

Tickets for Sage 10.2 (2023)

Tickets for Sage 10.3 (2023–2024)

Tickets for Sage 10.4 (2024)

See also: Modularization changes in Sage 10.4 (release notes)

Tickets for Sage 10.5 (2024)

See also: Modularization changes in Sage 10.5

Tickets for Sage 10.x (2024)

Wishlist tickets, loose ends

Related:

CC: @videlec @orlitzky @kiwifb @isuruf @tscrim @kliem @jhpalmieri @tobiasdiez @defeo @thierry-FreeBSD @anneschilling

Component: refactoring

Keywords: sd109, sd110, sd111

Issue created by migration from https://trac.sagemath.org/ticket/29705

mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -1,6 +1,6 @@
 (From https://groups.google.com/d/msg/sage-devel/M9QTWtln6zU/UHwkrmTKBQAJ)

-Modularize sagelib into separate distutils packages, so that these can be built and run with a much smaller set of libraries. The packages can be made available individually on !PyPI. The goal would be to do this **without changing the overall structure of the source tree**.
+Modularize sagelib into separate distutils packages, so that these can be built and run with a much smaller set of libraries. The packages can be made available individually on PyPI. The goal would be to do this **without changing the overall structure of the source tree**.

 1. In particular, create a package `sage_objects` that makes Sage objects, the element/parent framework, categories, the coercion system and the related metaclasses available. This package would, for example, not provide or know about integers, polynomials, or matrices (thus avoiding all the dependencies on the various libraries implementing these). Ideally it would only have a build dependency on Cython and no runtime dependencies.
mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -6,7 +6,7 @@

    (Doctesting `sage_objects` would still need more of Sage, but some it could also be done by mocking instead of with the real classes.)

-   a) These facilities could see a wider use in the mathematical Python community, and in this way we could hope to benefit from a larger developer base. For example, we would hope that `sage_objects` could be ported for use with PyPy instead of !CPython.
+   a) These facilities could see a wider use in the mathematical Python community, and in this way we could hope to benefit from a larger developer base. For example, we would hope that `sage_objects` could be ported for use with PyPy instead of CPython.

    b) Building and deploying a user package would be easier if it could depend on a small package such as `sage_objects` only instead of the whole multigigabyteness of Sage.
mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -17,7 +17,7 @@

 **Not** within the scope of this ticket:

-- We will not change the overall structure of the source tree (`SAGE_ROOT/src/sage`). We will achieve this by using the mechanism of native namespace packages introduced in Python 3.3 (see https://github.com/sagemath/sage-prod/issues/28925).
+- We will not change the overall structure of the source tree (`SAGE_ROOT/src/sage`). We will achieve this by using the mechanism of native namespace packages introduced in Python 3.3 (PEP 420, see https://github.com/sagemath/sage-prod/issues/28925).

 - It is not a prerequisite to fight against all circular imports. In particular, a parent class and its corresponding element class would certainly not be separated into separate distutils packages.
mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -45,6 +45,7 @@

 - #29411: make sagelib a script package
 - #29702: Move all code from `src/setup.py` to `sage_setup`
+- #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
 - #21516: Fix sagelib sdist (src/setup.py sdist)
 - #29701: Meta-ticket: Replace use of `OptionalExtension` by namespace packages
mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -45,6 +45,7 @@

 - #29411: make sagelib a script package
 - #29702: Move all code from `src/setup.py` to `sage_setup`
+- #29706: Move `Extension` options from `src/module_list.py` to `distutils:` directives in the individual files
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
 - #21516: Fix sagelib sdist (src/setup.py sdist)
 - #29701: Meta-ticket: Replace use of `OptionalExtension` by namespace packages
mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -52,3 +52,4 @@

 **Related:**
 - #21508 Meta-ticket: Clean up `src/setup.py` to bring it to standard distutils behavior
+- https://wiki.sagemath.org/CodeSharingWorkflow
mkoeppe commented 4 years ago

Changed keywords from none to sd109

mkoeppe commented 4 years ago

Branch: u/mkoeppe/META-modularize-sagelib

mkoeppe commented 4 years ago

Last 10 new commits:

47cb578is_package_or_namespace_package_dir: Recognize namespace packages by file 'namespace'
381a507src/module_list.py: Remove explicit list of sage.numerical.backends extensions
4c51133sage_setup/command/sage_install.py: WIP: Disable cleaning
c0a2ed7src/module_list.py: Remove explicit listing of sage.graphs.graph_decompositions extensions
acaa409Merge branch 't/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_' into HEAD
6909e23build/pkgs/coxeter3/distros/fedora.txt: New
2b060aesrc/module_list.py: Move OptionalExtension options for sage.libs.coxeter3.coxeter to distutils directive
4f0d07dsrc/sage/libs/coxeter3/decl.pxd: Add coxeter/ prefix to all header includes; src/module_list.py: remove include_dirs option
1497f3fbuild/pkgs/coxeter3/spkg-configure.m4: New
06a5911Merge branch 't/29721/spkg_configure_m4_for_coxeter3' into HEAD
mkoeppe commented 4 years ago

Commit: 06a5911

mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -45,10 +45,10 @@

 - #29411: make sagelib a script package
 - #29702: Move all code from `src/setup.py` to `sage_setup`
-- #29706: Move `Extension` options from `src/module_list.py` to `distutils:` directives in the individual files
+- #29706, #29720, #29721, #29785, #29786: Move `Extension` options from `src/module_list.py` to `distutils:` directives in the individual files
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
 - #21516: Fix sagelib sdist (src/setup.py sdist)
-- #29701: Meta-ticket: Replace use of `OptionalExtension` by namespace packages
+- #29701 (#28925): Meta-ticket: Replace use of `OptionalExtension` by namespace packages

 **Related:**
 - #21508 Meta-ticket: Clean up `src/setup.py` to bring it to standard distutils behavior
mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -45,7 +45,7 @@

 - #29411: make sagelib a script package
 - #29702: Move all code from `src/setup.py` to `sage_setup`
-- #29706, #29720, #29721, #29785, #29786: Move `Extension` options from `src/module_list.py` to `distutils:` directives in the individual files
+- #29706, #29720, #29721, #29785, #29786, #29790: Move `Extension` options from `src/module_list.py` to `distutils:` directives in the individual files
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
 - #21516: Fix sagelib sdist (src/setup.py sdist)
 - #29701 (#28925): Meta-ticket: Replace use of `OptionalExtension` by namespace packages
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 06a5911 to 4365e5d

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

55bd7basrc/module_list.py: Move options for Extensions in sage.graphs to distutils directives
4365e5dMerge branch 't/29790/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_5__sage_graphs_' into t/29705/META-modularize-sagelib
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

7d94648src/module_list.py: Move options for Extensions in sage.libs to distutils directives
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 4365e5d to 7d94648

mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -45,7 +45,7 @@

 - #29411: make sagelib a script package
 - #29702: Move all code from `src/setup.py` to `sage_setup`
-- #29706, #29720, #29721, #29785, #29786, #29790: Move `Extension` options from `src/module_list.py` to `distutils:` directives in the individual files
+- #29706, #29720, #29721, #29785, #29786, #29790, #29791: Move `Extension` options from `src/module_list.py` to `distutils:` directives in the individual files
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
 - #21516: Fix sagelib sdist (src/setup.py sdist)
 - #29701 (#28925): Meta-ticket: Replace use of `OptionalExtension` by namespace packages
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 7d94648 to b3d3d2f

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

9dc7022Merge branch 't/29706/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files' into t/29791/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_6__last_
f78b06dsrc/module_list.py: Move options for Extensions in sage.libs to distutils directives
1b0e29dsrc/module_list.py: Move options for Extensions in sage.matrix to distutils directives
6421e2csrc/module_list.py: Move remaining options for Extensions in sage.libs, sage.rings to distutils directives
b3d3d2fMerge branch 't/29791/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_6__last_' into t/29705/META-modularize-sagelib
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from b3d3d2f to 891d12a

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

2821934Fix sage_setup directives: Use distribution, not package
9052db4Merge branch 't/29720/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_2___optionalextensions_' into t/29705/META-modularize-sagelib
ff710eesrc/sage_setup/optional_extension.py (is_package_installed_and_updated): Factor out from OptionalExtension
891d12asrc/setup.py: Remove use of module_list.py; filter by distributions
mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -11,6 +11,11 @@
    b) Building and deploying a user package would be easier if it could depend on a small package such as `sage_objects` only instead of the whole multigigabyteness of Sage.

 2. Remove the mechanism of `OptionalExtension`s from `sagelib`.
+
+   Currently, a user would install, for example, the optional package `tdlib` and then rebuild `sagelib` so that the `OptionalExtension` `sage.graphs.graph_decompositions.tdlib` is built and installed.
+
+   With this ticket, the user would instead install a new optional package `sage-tdlib` (which has `tdlib` as a dependency); this installs the `Extension` `sage.graphs.graph_decompositions.tdlib` (as a namespace package).
+

    This is #29701 Meta-ticket: Replace use of `OptionalExtension` by namespace packages

@@ -46,9 +51,9 @@
 - #29411: make sagelib a script package
 - #29702: Move all code from `src/setup.py` to `sage_setup`
 - #29706, #29720, #29721, #29785, #29786, #29790, #29791: Move `Extension` options from `src/module_list.py` to `distutils:` directives in the individual files
+- #29701 (#28925): Replace use of `module_list` and `OptionalExtension` by extending `sage_setup.find_python_sources`
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
 - #21516: Fix sagelib sdist (src/setup.py sdist)
-- #29701 (#28925): Meta-ticket: Replace use of `OptionalExtension` by namespace packages

 **Related:**
 - #21508 Meta-ticket: Clean up `src/setup.py` to bring it to standard distutils behavior
mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -58,3 +58,10 @@
 **Related:**
 - #21508 Meta-ticket: Clean up `src/setup.py` to bring it to standard distutils behavior
 - https://wiki.sagemath.org/CodeSharingWorkflow
+
+
+
+The branch on this meta-ticket is not intended for merging.
+It is an integration branch to facilitate testing. See https://github.com/mkoeppe/sage/pull/35 for automated tests
+
+
mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -15,9 +15,6 @@
    Currently, a user would install, for example, the optional package `tdlib` and then rebuild `sagelib` so that the `OptionalExtension` `sage.graphs.graph_decompositions.tdlib` is built and installed.

    With this ticket, the user would instead install a new optional package `sage-tdlib` (which has `tdlib` as a dependency); this installs the `Extension` `sage.graphs.graph_decompositions.tdlib` (as a namespace package).
-
-
-   This is #29701 Meta-ticket: Replace use of `OptionalExtension` by namespace packages

 **Not** within the scope of this ticket:
mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -48,8 +48,9 @@
 - #29411: make sagelib a script package
 - #29702: Move all code from `src/setup.py` to `sage_setup`
 - #29706, #29720, #29721, #29785, #29786, #29790, #29791: Move `Extension` options from `src/module_list.py` to `distutils:` directives in the individual files
-- #29701 (#28925): Replace use of `module_list` and `OptionalExtension` by extending `sage_setup.find_python_sources`
+- #29701 (depends on all of the above): Replace use of `module_list` and `OptionalExtension` by extending `sage_setup.find_python_sources`
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
+- #28925 Modify find_python_sources, clean_stale_files to support modularization of sagelib by native namespace packages (PEP 420)
 - #21516: Fix sagelib sdist (src/setup.py sdist)

 **Related:**
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

0295c8fsrc/module_list.py: Move options for Extensions in sage.graphs.graph_decompositions to distutils directives
b582789src/sage_setup/find.py: Filter by directive 'sage_setup: distribution = PKG', find Cython modules
63c64d5Merge branches 't/29720/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_2___optionalextensions_', 't/29721/spkg_configure_m4_for_coxeter3', 't/29785/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_3__get_rid_of_uname_specific_', 't/29786/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_4__sage_rings_', 't/29790/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_5__sage_graphs_' and 't/29791/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_6__last_' into t/29701/replace_use_of_module_list_optionalextension
ae70c81src/sage_setup/optional_extension.py (is_package_installed_and_updated): Factor out from OptionalExtension
ec7e9c5src/setup.py: Remove use of module_list.py; filter by distributions
bd26ddfMerge branch 't/29701/replace_use_of_module_list_optionalextension' into t/29705/META-modularize-sagelib
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 891d12a to bd26ddf

mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -51,6 +51,7 @@
 - #29701 (depends on all of the above): Replace use of `module_list` and `OptionalExtension` by extending `sage_setup.find_python_sources`
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
 - #28925 Modify find_python_sources, clean_stale_files to support modularization of sagelib by native namespace packages (PEP 420)
+- Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`; or use our `sage_install` command, which installs in the same format.
 - #21516: Fix sagelib sdist (src/setup.py sdist)

 **Related:**
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from bd26ddf to 26b5575

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

38b6bcfMerge tag '9.2.beta0' into t/29411/make_sagelib_a_script_package
f9a30f6build/pkgs/sagelib/spkg-install: Fix up error exits
00a1d57Merge branch 't/29411/make_sagelib_a_script_package' into t/29702/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup
25a2340Merge branch 't/29702/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup' into t/29701/replace_use_of_module_list_optionalextension
d7bd69eMerge branch 't/29701/replace_use_of_module_list_optionalextension' into t/29705/META-modularize-sagelib
2d866a3src/sage/graphs/graph_decompositions/tdlib.pyx: Fixup
14f488fMerge branch 't/29701/replace_use_of_module_list_optionalextension' into t/29705/META-modularize-sagelib
0d9f653WIP: Add sage_tdlib package
26b5575Use sage_install
mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -52,6 +52,7 @@
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
 - #28925 Modify find_python_sources, clean_stale_files to support modularization of sagelib by native namespace packages (PEP 420)
 - Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`; or use our `sage_install` command, which installs in the same format.
+- Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #21516: Fix sagelib sdist (src/setup.py sdist)

 **Related:**
mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -50,7 +50,9 @@
 - #29706, #29720, #29721, #29785, #29786, #29790, #29791: Move `Extension` options from `src/module_list.py` to `distutils:` directives in the individual files
 - #29701 (depends on all of the above): Replace use of `module_list` and `OptionalExtension` by extending `sage_setup.find_python_sources`
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
-- #28925 Modify find_python_sources, clean_stale_files to support modularization of sagelib by native namespace packages (PEP 420)
+- #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420)
+- #29803: Upgrade `setuptools`, `setuptools_scm`, `pip` (2020-06), add package `wheel`; remove `zope_interface`
+- #13190: make `sagelib` use `setuptools` instead of `distutils`
 - Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`; or use our `sage_install` command, which installs in the same format.
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #21516: Fix sagelib sdist (src/setup.py sdist)
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

29327d3build/pkgs/entrypoints: Do not provide setup.py, not needed for entrypoints 0.3
bcc79a5build/pkgs/entrypoints/spkg-install.in: Remove pyproject.toml so that setup.py is used
361b98bbuild/pkgs/terminado/spkg-install.in: Remove pyproject.toml so that setup.py is used
5040d7ebuild/pkgs/testpath/spkg-install.in: Remove pyproject.toml so that setup.py is used
371ede6build/pkgs/zope_interface: Change to a dummy script package
b7c63dfMerge branch 't/29803/upgrade_setuptools__pip__2020_06_' into t/29705/META-modularize-sagelib
5f33436src/setup.py: from setuptools import setup
93bbbfeAdd src/setup.cfg
a7de6c0Merge branch 't/13190/make_sagelib_use_setuptools_instead_of_distutils' into t/29705/META-modularize-sagelib
7dd469fWIP: Add sage_tdlib package, use symlinks for sagelib package
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 26b5575 to 7dd469f

mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -53,6 +53,7 @@
 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420)
 - #29803: Upgrade `setuptools`, `setuptools_scm`, `pip` (2020-06), add package `wheel`; remove `zope_interface`
 - #13190: make `sagelib` use `setuptools` instead of `distutils`
+- #29845: PEP 517 `buildapi` for `sage_setup`
 - Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`; or use our `sage_install` command, which installs in the same format.
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #21516: Fix sagelib sdist (src/setup.py sdist)
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 7dd469f to 24e9b30

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

24e9b30build/pkgs/sage_tdlib/src/MANIFEST.in: New
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 24e9b30 to 9b59435

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

9b59435fixup symlinks
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 9b59435 to 19e061d

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

60ed86csrc/setup.py: WIP: Do not use sage_install
19e061dbuild/pkgs/sage_tdlib/src/sage_setup: Remove, we use the installed one
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 19e061d to c3593b2

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

c3593b2build/pkgs/sage_tdlib/src/sage_setup: Remove, we use the installed one
mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -24,6 +24,8 @@
 - It is not a prerequisite to fight against all circular imports. In particular, a parent class and its corresponding element class would certainly not be separated into separate distutils packages. 

 **Constraints and challenges:**
+
+- Inconsistent handling of symlinks by `pip`/`setuptools`. We cannot simply create a distribution of a subset of the files in `src` by creating a flock of symlinks (see #29850)

 - The category framework can not really be separated from the `Parent`/`Element`/`CategoryObject` (e.g. coercion is involved in comparisons and binary operations).
mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -25,7 +25,7 @@

 **Constraints and challenges:**

-- Inconsistent handling of symlinks by `pip`/`setuptools`. We cannot simply create a distribution of a subset of the files in `src` by creating a flock of symlinks (see #29850)
+- Inconsistent handling of symlinks by `pip`/`setuptools`. We cannot simply create a distribution of a subset of the files in `src` by creating a flock of symlinks (see #29850) and expect everything to work; notably, pip fails because of longstanding issues with installs from local directories (see discussion in #29854).

 - The category framework can not really be separated from the `Parent`/`Element`/`CategoryObject` (e.g. coercion is involved in comparisons and binary operations).
mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -26,6 +26,8 @@
 **Constraints and challenges:**

 - Inconsistent handling of symlinks by `pip`/`setuptools`. We cannot simply create a distribution of a subset of the files in `src` by creating a flock of symlinks (see #29850) and expect everything to work; notably, pip fails because of longstanding issues with installs from local directories (see discussion in #29854).
+
+  Solution: Do not use pip for installations from local source directories.  Use `setup.py install` (distutils) directly; or use `setup.py sdist`, followed by `pip install` of the sdist; or use a custom PEP 517 backend (#29845) for making an sdist, followed by `pip install` of the sdist.

 - The category framework can not really be separated from the `Parent`/`Element`/`CategoryObject` (e.g. coercion is involved in comparisons and binary operations).
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from c3593b2 to 458d851

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

2612091Use setup.py install --single-version-externally-managed
458d851build/pkgs/sage_tdlib/spkg-src: New
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 458d851 to 7042564

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

5d4e8e7build/pkgs/sage_tdlib/dependencies: Add sagelib, python toolchain
7042564build/pkgs/sagelib/spkg-src: New