Meta-ticket: Modularize sagelib into separate distributions (pip-installable packages) sagemath-... for Sage 10.x

[mkoeppe]

We modularize sagelib into separate distributions (pip-installable packages), so that these can be built and run with a much smaller set of dependencies. The packages are made available individually on PyPI in source and wheel form. The modularization is done without changing the overall structure of the source tree by means of PEP 420 implicit namespace packages.

• SageMath modularization project: The five by five (sage-devel, 2023-06) I. The goals – II. The tools – III. The hooks – IV. The rules – V. The blocs
• Video from Sage Days 120
• Section Packaging the Sage Library in the Sage Developer's Guide (added in Sage 9.5)

Mega-PR #35095 (not for merging; changes are being cherry-picked from here for integration in Sage) adds the new distributions. Instead of defining a distribution package for every possible community or subfield of mathematics that Sage supports, the modularization project introduces three types of distribution packages:

Distributions named after a basic mathematical structure: The packages may also cover a wide range of generalizations/applications of the structure after which they are named. Users who work in a specialized research area will, of course, recognize what structures they need. The down-to-earth naming also creates discoverability by a broader audience. Not many more distribution packages than these 7 are needed:

• sagemath-combinat: provides "everything combinatorial", except for graphs.
• sagemath-graphs: also provides posets, combinatorial designs, abstract simplicial complexes, quivers, etc.
• sagemath-groups
• sagemath-modules: also has matrices, tensors, homology, coding theory, etc.
• sagemath-polyhedra: also provides fans, hyperplane arrangements, polyhedral complexes, linear and mixed-integer optimization, lattice point sets, and toric varieties
• sagemath-schemes
• sagemath-symbolics: symbolic expressions and symbolic calculus

Distributions named after a third-party non-Python dependency: This makes technical sense because the dependencies will be localized to this distribution package, but it also helps give attribution and visibility to these libraries and projects that Sage depends on.	Standard packages	Optional Packages
sagemath-flint	sagemath-bliss
sagemath-gap	sagemath-brial
sagemath-glpk	sagemath-coxeter3
sagemath-giac	sagemath-mcqd
sagemath-homfly	sagemath-meataxe
sagemath-lcalc	sagemath-sirocco
sagemath-libbraiding	sagemath-tdlib
sagemath-libecm
sagemath-linbox
sagemath-ntl
sagemath-pari
sagemath-singular

Distributions named after a technical functionality:

• sagemath-objects: Sage extends Python's object system by dynamic mix-in classes that are driven by categories and axioms. It is loosely modeled on concepts of category theory and inspired by Scratchpad/Axiom/FriCAS, Magma, and MuPAD. This distribution package makes Sage objects, the element/parent framework, basic categories and functors, the coercion system and the related metaclasses available.
• sagemath-categories: This distribution package contains the full set of categories defined by Sage, as well as basic mathematical objects such as integers and rational numbers, a basic implementation of polynomials, and affine spaces. None of this brings in additional dependencies.
• sagemath-environment
• sagemath-plot: Plotting facilities, depending on matplotlib.
• sagemath-repl: The top-level interactive environment with the preparser that defines the surface language of Sage. This distribution also includes the doctesting facilities, as the doctests are written in the surface language.
• sagemath-standard: Everything as provided by a standard installation of the Sage distribution. This is reduced to an empty meta-package.
• sagemath-standard-no-symbolics

---

Older posts and presentations:

• Initial proposal (sage-devel, 2020-05)
• Presentation video: https://researchseminars.org/talk/SageDays110/11/ (2020-10)
• Schematics (2020-10)
• Current status of pip-installability (Sage 9.4 release tour, 2021-08)
• Refactoring tasks in the Sage library to support modularization (sage-devel, 2021-09)
• How to modularize for fun and profit (sage-devel, 2021-10)
  • I. Namespace vs. distribution name
  • II. MONOREPO vs. MULTIREPO
  • III. Dependencies, in 4 special flavors
  • IV. Testing subset distributions in virtual envs
• Master plan of sage modularization (sage-devel, 2021-11)
• Presentation video: https://researchseminars.org/talk/SageDays112358/17/ (2022-06)

---

Initial steps: (2020)

1. In particular, create a package sagemath-objects that makes Sage objects, the element/parent framework, categories, the coercion system and the related metaclasses available. This package would, for example, not provide or know about integers, polynomials, or matrices (thus avoiding all the dependencies on the various libraries implementing these). Ideally it would only have a build dependency on Cython and no runtime dependencies.

   (Doctesting sagemath-objects would still need more of Sage, but some it could also be done by mocking instead of with the real classes.)

   a) These facilities could see a wider use in the mathematical Python community, and in this way we could hope to benefit from a larger developer base. For example, we would hope that sagemath-objects could be ported for use with PyPy instead of CPython.

   b) Building and deploying a user package would be easier if it could depend on a small package such as sagemath-objects only instead of the whole multigigabyteness of Sage.

   This has been completed in Sage 9.6 (#29865).

2. Remove the mechanism of OptionalExtensions from sagelib.

   Currently, a user would install, for example, the optional package tdlib and then rebuild sagelib so that the OptionalExtension sage.graphs.graph_decompositions.tdlib is built and installed.

   With this ticket, the user would instead install a new optional package sagemath-tdlib (which has tdlib as a dependency); this installs the Extension sage.graphs.graph_decompositions.tdlib (as a namespace package).

   This has been completed in Sage 9.2 (#29701).

Not within the scope of this ticket:

• We will not change the overall structure of the source tree (SAGE_ROOT/src/sage). We will achieve this by using the mechanism of native namespace packages introduced in Python 3.3 (PEP 420, see https://github.com/sagemath/sage-prod/issues/28925).

• It is not a prerequisite to fight against all circular imports. In particular, a parent class and its corresponding element class would certainly not be separated into separate distribution packages.

Initially noted constraints and challenges: (2020)

• Inconsistent handling of symlinks by pip/setuptools. We cannot simply create a distribution of a subset of the files in src by creating a flock of symlinks (see #29850) and expect everything to work; notably, pip fails because of longstanding issues with installs from local directories (see discussion in #29854).

  Solution (implemented in Sage 9.2, #29500): Do not use pip for installations from local source directories. Use setup.py bdist_wheel, followed by pip install of the wheel. pip 21.1's --use-feature=in-tree-build makes this workaround unnecessary (#32046, Sage 9.5).

• The category framework can not really be separated from the Parent/Element/CategoryObject (e.g. coercion is involved in comparisons and binary operations).

  Solution: Therefore they are in the same distribution sagemath-objects.

• The category framework cannot live without the integers (e.g. the output of .cardinality() expects an integer).

  Near term solution: #32432 makes sagemath-categories larger, to include integers and some other basic rings.

  Possible future solution: The coercion system already has the notion of a global coercion model - which is determined at runtime, not compile time (sage.structure.element.get_coercion_model). When methods in sagemath-objects need to create an integer, they should obtain the appropriate class of integers in the same way. An existing function provides this interface already (but has a static implementation):

  sage: py_scalar_parent(int)
  Integer Ring

  29875 will make this implementation dynamic.

• Imports in sage.categories:

  bimodules imports QQ, RR
  classical_crystals imports SR
  finite_coxeter_groups imports QQbar

  Such imports can be removed using the techniques described in section Module-level runtime dependencies in the Sage developer's guide.

Tickets for Sage 9.2: (2020)

see also Modularization changes in Sage 9.2 (release tour)

• 29411

• 29702

• Move Extension options from src/module_list.py to distutils: directives in the individual files: #29706, #29720, #29721, #29785, #29786, #29790, #29791

• 29701 (depends on all of the above)

• 28197 (old ipython completer crashes on namespace packages)

• 29803

• 21559

• 29950 (supersedes: #21516 / #13190)

• Improve/repair filtered structure of sagelib: #29869, #29873, #29892, #29883, #16351, #29881, #29880, #29916, #29922

• 29171

• 30332

• 29500

• 30606

• 30616

Tickets for Sage 9.3: (2021)

see also Modularization changes in Sage 9.3 (release tour)

• 30715

• 30739

• 30784

• 30779

• 30780

• 30579

• 30580

• 29355

• 30780

• 30719

• 29951

• 29852

• 22731

• 30013

• 30587

• 31261

• 29850:

• 31025

• 31058

• 31218

• 31270

• 31321

• sagemath-standard: #30912 / #31357

• 31332

• 31334

• 30984

• 31365

• 31279

• 31362

• 31377

• 31045

• sage-docbuild: #30010

• 31278

• 30383

• 31523

• 31409

Tickets for Sage 9.4: (2021)

see also Modularization changes in Sage 9.4 (release tour)

• 30913 (supersedes #30578, #30581)

• 31384

• 29013

• 31593

• 31338

• 31562

• 28890

• 32045

• 32036

• 32057

• 31577

• 31333

• 32073

Tickets for Sage 9.5: (2022)

see also Modularization changes in Sage 9.5 (release tour)

• 32046

• 29847. Deploy distribution sage-setup to PyPI.

• 32371

• 31031

• 32386 (supersedes #30534)

• 32412 ... and sage.libs.primecount

• 32373

• 32501:

  • Review/remove monkey patching in src/sage/__init__.py: #31420 / #32479 / #32489
  • __init__.py in other packages: #32506 / #32507 / #32508 / #32509

  • 32840

• 32414:

  • sage.rings.abc: #32566 / #32610 / #32612 / #32660 / #32750

  • 32600

  • 32606

  • 32742

• Remove .all imports: #32534 / #32591 / #32620 / #32734 / #32733

• 32442

• 32062

• 29039

• 32601: <!--- Modularization of sagelib: Break out a separate package sagemath-standard-no-symbolics --->

  • Remove unnecessary uses of SR and symbolic functions: #32411 / #32413 / #32415 / #32416

  • 32599

  • 32593

  • 32638

  • 32665

  • 32708

  • 32712

  • 32725

  • 32730

  • 32717

  • 32731

• 32585

• 32614

• 30778

• 32649

• 32650

• 32619

• 32432: <!--- Modularization of sagelib: Break out a separate package sagemath-polyhedra --->

  • Remove compile-time dependency on cypari2 and flint from some modules: #30022 / #32441 / #32455 / #32775 / #32701

  • 32592

  • 32628

  • 32634

  • sage.geometry.polyhedron: Mark doctests # optional - sage.rings.number_field etc.: #32652 / #32653 / #32732

  • 32635

  • 32666

  • 32677

  • 32625

  • 32641

  • 32769

  • 32777

  • 32780

  • 32778

• 31386

• 32899

• Split out sage.libs.primecount/sage.interfaces.primecount as a separate Python library: #25009 / #32894:

Tickets for Sage 9.6: (2022)

see also Modularization changes in Sage 9.6 (release tour)

• 29865: These distributions only depend on Cython, pkgconfig, cysignals, gmpy2. Use them as a testsuite for testing the filtered structure of sagelib.

• 31013

• 32846

• 32847

• 31306

• Features: #32881 / #32888

• 33033

• 32465

• 33402

• 33403

• 33392 (#33159)

• Remove imports from sage.*.all: #32989 / #32999 / #33000 / #33007 / #33146 / #33199

• 32609

• 32893

• 31296

• 33440

• Use Executable.absolute_filename: #33465, #33466, #32645, #33467

• 33017

Tickets for Sage 9.7: (2022)

see also Modularization changes in Sage 9.7 (release tour)

• Remove some uses of SAGE_TMP: #33797 / #33799 (supersedes #32986)

• 33801

• 33803

• 32716

• 29941

• 28925, rework sagemath-objects, sagemath-categories as namespace packages

• 33822 python3 spkg-configure.m4

• 33795

• 33821

• 33295

• 33029

• 33800

• 33011

• 34187

• 32406

• 33817

• 34221

• Remove .all imports: #34189 / #34190 / #34191 / #34192 / #34428

• 34296

Tickets for Sage 9.8: (2023)

see also Modularization changes in Sage 9.8 (release tour)

• 33812

• 32874 (supersedes #32927, #30152)

• Deprecate sage.interfaces is_...Element functions: #34770/#34823/#34804

• 16522

• 34547

• 34858

Tickets for Sage 10.0 (2023)

• 34945

• Remove .all imports: #34946 / #34947 / #34948 / #34949 / #34951 / #34952 / #34953 / #34954 / #34955 / #34956 / #34957 / #34201 / #35372

• 35135

• 35100

• 34998

• 32664

• 32709

• 35240

• 35253

• 35322 / #35366

• 35237

• Modularization fixes: #34855 / #35153 / #35229 / #35230 / #35243 / #35263 / #35279 / #35283 / #35305 / #35306 / #35389
• Add # optional: #35136 / #35266 / #35267 / #35314

Tickets for Sage 10.1 (2023)

• 35457

• 35502

• 35554

• 35581

• 35586

• 35647

• 35758

• 35780

• 35728

• 35620

• 35564

• 32433

• 32721

• 35718

• 35719

• 35729

• 35734

• 35741

• 35401

• 34346

• 35717

• 36024

• 36026

• 35881

• 35661

• 35919

• 35847

• 35943

• 35951

• 35957

• 36003

• 36016

• 36015

• 35749

• 35884

• 36052

• 36033

• 36077

• 36037

• 36045

• 36044

• 36038

• 36056

Tickets for Sage 10.2 (2023)

• 36114

• 36106

• 36089

• 36067

• 36057

• 35945

• 35716

• 36152

• 36102

• 36168

• 36307

• 36264

• 36272

• 36452

• 36304

• 36520

• 36495

• 36521

• 36532

• 36271

• 36259

• 36563

• 36567

• 36568

• 36569

• 36594

• 36595

• 36596

• 36598

• 36605

• 36612

• 36618

• 36619

Tickets for Sage 10.3 (2023–2024)

• 36917

• 36905

• 36904

• 36950

• 36948

• 36865

• 36900

• 36640

• 36747

• 36859

• 36564

• 36645

• 36656

• 36666

• 36679

• 36738

• 36743

• 36135

• 36642

• 36562

• 36572

• 36589

• 36597

• 36642

• 36643

• 36644

• 36657

• 36658

• 35838/#33575

Tickets for Sage 10.4 (2024)

• 37083

• 37720

• 37022

• 37083

• 36676 / #36951 / #36964 / #37796

• 35618

• 36498

• 37750

• 37057

• 37715

• 37486

• 37720

• 37022

Tickets for Sage 10.x (2024)

• Mega-PR: #35095

• 37900

• 37901

• 37902

• 30666

• 33577

• 31444

• 33037

• 32432

• 31396

• 32601

  • 32718

• 32586

• 34633

• 34209

• 34631

• 34587

• 33820

• 33802

• 32360

  • 32431

• 29870

• 30746

• 30848

• 30911

• possibly move pkgs/sage-conf/sage_conf.py[.in] into src/, mark it (and also src/bin/sage-env-config[.in]) # sage_setup: distribution = sage-conf.

• 29868

• 31543

• 30647

• 33405

• 30036

• 29041

• Further modularization:

  • 32700

  • sagemath-brial (ex sage-brial in sage.rings.polynomial.pbori + sage.libs.polybori, sage.crypto.boolean_function)

  • 30716

  • 33660

  • 32702

  • 29912

  • 31695

• 32613

• 32908

• 30371

Wishlist tickets, loose ends

• 30527

• 29877

• 29875

• 29874

• 29845

• 31602

• 30748

• 30896

Related:

• 21508

• https://wiki.sagemath.org/CodeSharingWorkflow

• 30914

• 30922

• 30818

  • 31041

• 31251

• 33580

• 34201

CC: @videlec @orlitzky @kiwifb @isuruf @tscrim @kliem @jhpalmieri @tobiasdiez @defeo @thierry-FreeBSD @anneschilling

Component: refactoring

Keywords: sd109, sd110, sd111

Issue created by migration from https://trac.sagemath.org/ticket/29705

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

90b33d8

Install sagelib, sage_tdlib via sdist defined by MANIFEST.in

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 7042564 to 90b33d8

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

2ccf6b7

Merge tag '9.2.beta1' into t/29705/META-modularize-sagelib

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 90b33d8 to 2ccf6b7

[mkoeppe]

Description changed:

--- 
+++ 
@@ -56,9 +56,10 @@
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420)
 - #29803: Upgrade `setuptools`, `setuptools_scm`, `pip` (2020-06), add package `wheel`; remove `zope_interface`
+- Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`, or use distutils, or disable egg building with setuptools using --single-version-externally-managed.
+   - #29864: Modularization of sagelib: Break out a separate package sage-tdlib
 - #13190: make `sagelib` use `setuptools` instead of `distutils`
 - #29845: PEP 517 `buildapi` for `sage_setup`
-- Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`; or use our `sage_install` command, which installs in the same format.
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #21516: Fix sagelib sdist (src/setup.py sdist)

[mkoeppe]

Description changed:

--- 
+++ 
@@ -2,13 +2,13 @@

 Modularize sagelib into separate distutils packages, so that these can be built and run with a much smaller set of libraries. The packages can be made available individually on PyPI. The goal would be to do this **without changing the overall structure of the source tree**.

-1. In particular, create a package `sage_objects` that makes Sage objects, the element/parent framework, categories, the coercion system and the related metaclasses available. This package would, for example, not provide or know about integers, polynomials, or matrices (thus avoiding all the dependencies on the various libraries implementing these). Ideally it would only have a build dependency on Cython and no runtime dependencies. 
+1. In particular, create a package `sage-objects` that makes Sage objects, the element/parent framework, categories, the coercion system and the related metaclasses available. This package would, for example, not provide or know about integers, polynomials, or matrices (thus avoiding all the dependencies on the various libraries implementing these). Ideally it would only have a build dependency on Cython and no runtime dependencies. 

-   (Doctesting `sage_objects` would still need more of Sage, but some it could also be done by mocking instead of with the real classes.)
+   (Doctesting `sage-objects` would still need more of Sage, but some it could also be done by mocking instead of with the real classes.)

-   a) These facilities could see a wider use in the mathematical Python community, and in this way we could hope to benefit from a larger developer base. For example, we would hope that `sage_objects` could be ported for use with PyPy instead of CPython.
+   a) These facilities could see a wider use in the mathematical Python community, and in this way we could hope to benefit from a larger developer base. For example, we would hope that `sage-objects` could be ported for use with PyPy instead of CPython.

-   b) Building and deploying a user package would be easier if it could depend on a small package such as `sage_objects` only instead of the whole multigigabyteness of Sage.
+   b) Building and deploying a user package would be easier if it could depend on a small package such as `sage-objects` only instead of the whole multigigabyteness of Sage.

 2. Remove the mechanism of `OptionalExtension`s from `sagelib`.

@@ -57,7 +57,8 @@
 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420)
 - #29803: Upgrade `setuptools`, `setuptools_scm`, `pip` (2020-06), add package `wheel`; remove `zope_interface`
 - Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`, or use distutils, or disable egg building with setuptools using --single-version-externally-managed.
-   - #29864: Modularization of sagelib: Break out a separate package sage-tdlib
+   - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib`
+   - #29865: Modularization of sagelib: Break out a separate package `sage-objects`
 - #13190: make `sagelib` use `setuptools` instead of `distutils`
 - #29845: PEP 517 `buildapi` for `sage_setup`
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.

[mkoeppe]

Description changed:

--- 
+++ 
@@ -35,7 +35,13 @@

 - The category framework cannot live without the integers (e.g. the output of `.cardinality()` expects an integer). 

-  Possible solution: The coercion system already has the notion of a global coercion model - which is determined at runtime, not compile time (`sage.structure.element.get_coercion_model`). When methods in `sage_objects` need to create an integer, they should obtain the appropriate class of integers in the same way.
+  Possible solution: The coercion system already has the notion of a global coercion model - which is determined at runtime, not compile time (`sage.structure.element.get_coercion_model`). When methods in `sage_objects` need to create an integer, they should obtain the appropriate class of integers in the same way. An existing function provides this interface already (but has a static implementation):
+
+   ```
+   sage: py_scalar_parent(int)
+   Integer Ring
+   ```
+  #29875 will make this implementation dynamic.

 - Imports in `sage.categories`:

@@ -63,6 +69,8 @@
 - #29845: PEP 517 `buildapi` for `sage_setup`
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #21516: Fix sagelib sdist (src/setup.py sdist)
+- #29874: Mock integer class
+- #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic

 **Related:**
 - #21508 Meta-ticket: Clean up `src/setup.py` to bring it to standard distutils behavior

[mkoeppe]

Description changed:

--- 
+++ 
@@ -71,6 +71,7 @@
 - #21516: Fix sagelib sdist (src/setup.py sdist)
 - #29874: Mock integer class
 - #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic
+- #29877: `sage.categories`: Replace import of `ZZ` by `py_scalar_parent(int)`

 **Related:**
 - #21508 Meta-ticket: Clean up `src/setup.py` to bring it to standard distutils behavior

[mkoeppe]

Description changed:

--- 
+++ 
@@ -65,6 +65,7 @@
 - Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`, or use distutils, or disable egg building with setuptools using --single-version-externally-managed.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib`
    - #29865: Modularization of sagelib: Break out a separate package `sage-objects`
+   - #29912: Modularization of sagelib: Break out a separate package `sage-ntl`
 - #13190: make `sagelib` use `setuptools` instead of `distutils`
 - #29845: PEP 517 `buildapi` for `sage_setup`
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.

[mkoeppe]

Description changed:

--- 
+++ 
@@ -64,7 +64,7 @@
 - #29803: Upgrade `setuptools`, `setuptools_scm`, `pip` (2020-06), add package `wheel`; remove `zope_interface`
 - Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`, or use distutils, or disable egg building with setuptools using --single-version-externally-managed.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib`
-   - #29865: Modularization of sagelib: Break out a separate package `sage-objects`
+   - #29865: Modularization of sagelib: Break out separate packages `sage-objects`, `sage-categories`
    - #29912: Modularization of sagelib: Break out a separate package `sage-ntl`
 - #13190: make `sagelib` use `setuptools` instead of `distutils`
 - #29845: PEP 517 `buildapi` for `sage_setup`

[mkoeppe]

Description changed:

--- 
+++ 
@@ -66,6 +66,7 @@
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib`
    - #29865: Modularization of sagelib: Break out separate packages `sage-objects`, `sage-categories`
    - #29912: Modularization of sagelib: Break out a separate package `sage-ntl`
+   - #29941: Modularization of sagelib: Break out a separate package `sage-repl`
 - #13190: make `sagelib` use `setuptools` instead of `distutils`
 - #29845: PEP 517 `buildapi` for `sage_setup`
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.

[mkoeppe]

Description changed:

--- 
+++ 
@@ -53,24 +53,31 @@

   One can try to not populate the whole category system.

-**Tickets and steps:**
+**Tickets for Sage 9.2:**

 - #29411: make sagelib a script package
 - #29702: Move all code from `src/setup.py` to `sage_setup`
 - #29706, #29720, #29721, #29785, #29786, #29790, #29791: Move `Extension` options from `src/module_list.py` to `distutils:` directives in the individual files
 - #29701 (depends on all of the above): Replace use of `module_list` and `OptionalExtension` by extending `sage_setup.find_python_sources`
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
+- #29803: Upgrade `setuptools`, `setuptools_scm`, `pip` (2020-06), add package `wheel`; remove `zope_interface`
+- #21559: Install `src/bin` scripts by sagelib's `setup.py`, not make
+- #29850: Install `sage-env-config` as part of `sage_conf`
+- Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
+- #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts"
+- #29950: Build sagelib using the installed `sage_setup`, add `spkg-src` (supersedes: #21516: Fix sagelib sdist (src/setup.py sdist))
+- #29865: Modularization of sagelib: Break out separate experimental packages `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
+- #29869, #29873, #29892, #29883, #16351, #29881, #29880, #29916, #29922: Improve/repair filtered structure of sagelib.
+- #13190: make `sagelib` use `setuptools` instead of `distutils`
+
+**Tickets for Sage 9.3:**
+
 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420)
-- #29803: Upgrade `setuptools`, `setuptools_scm`, `pip` (2020-06), add package `wheel`; remove `zope_interface`
 - Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`, or use distutils, or disable egg building with setuptools using --single-version-externally-managed.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib`
-   - #29865: Modularization of sagelib: Break out separate packages `sage-objects`, `sage-categories`
    - #29912: Modularization of sagelib: Break out a separate package `sage-ntl`
    - #29941: Modularization of sagelib: Break out a separate package `sage-repl`
-- #13190: make `sagelib` use `setuptools` instead of `distutils`
 - #29845: PEP 517 `buildapi` for `sage_setup`
-- Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
-- #21516: Fix sagelib sdist (src/setup.py sdist)
 - #29874: Mock integer class
 - #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic
 - #29877: `sage.categories`: Replace import of `ZZ` by `py_scalar_parent(int)`

[mkoeppe]

Description changed:

--- 
+++ 
@@ -74,6 +74,7 @@

 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420)
 - Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`, or use distutils, or disable egg building with setuptools using --single-version-externally-managed.
+   - Rework `sage-objects`, `sage-categories`, and `sagelib` as namespace packages.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib`
    - #29912: Modularization of sagelib: Break out a separate package `sage-ntl`
    - #29941: Modularization of sagelib: Break out a separate package `sage-repl`

[mkoeppe]

Description changed:

--- 
+++ 
@@ -77,7 +77,9 @@
    - Rework `sage-objects`, `sage-categories`, and `sagelib` as namespace packages.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib`
    - #29912: Modularization of sagelib: Break out a separate package `sage-ntl`
+   - Modularization of sagelib: Break out a separate package `sage-flint-arb-e_antic`
    - #29941: Modularization of sagelib: Break out a separate package `sage-repl`
+   - Modularization of sagelib: Break out a separate package `sage-maxima` (depends on ecl and maxima, packages `sage.interfaces.maxima*`, `sage.libs.ecl`)
 - #29845: PEP 517 `buildapi` for `sage_setup`
 - #29874: Mock integer class
 - #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic

[mkoeppe]

Description changed:

--- 
+++ 
@@ -65,7 +65,7 @@
 - #29850: Install `sage-env-config` as part of `sage_conf`
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts"
-- #29950: Build sagelib using the installed `sage_setup`, add `spkg-src` (supersedes: #21516: Fix sagelib sdist (src/setup.py sdist))
+- #29950: Build sagelib from build/pkgs/sagelib/src, fix `setup.py sdist`, add `spkg-src` and `tox.ini` (supersedes: #21516: Fix sagelib sdist (src/setup.py sdist))
 - #29865: Modularization of sagelib: Break out separate experimental packages `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
 - #29869, #29873, #29892, #29883, #16351, #29881, #29880, #29916, #29922: Improve/repair filtered structure of sagelib.
 - #13190: make `sagelib` use `setuptools` instead of `distutils`

[mkoeppe]

Description changed:

--- 
+++ 
@@ -76,10 +76,11 @@
 - Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`, or use distutils, or disable egg building with setuptools using --single-version-externally-managed.
    - Rework `sage-objects`, `sage-categories`, and `sagelib` as namespace packages.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib`
+   - #30151: Modularization of sagelib: Break out a separate package `sage-meataxe`
    - #29912: Modularization of sagelib: Break out a separate package `sage-ntl`
    - Modularization of sagelib: Break out a separate package `sage-flint-arb-e_antic`
    - #29941: Modularization of sagelib: Break out a separate package `sage-repl`
-   - Modularization of sagelib: Break out a separate package `sage-maxima` (depends on ecl and maxima, packages `sage.interfaces.maxima*`, `sage.libs.ecl`)
+   - Modularization of sagelib: Break out a separate package `sage-maxima` (depends on ecl and maxima, packages `sage.interfaces.maxima*`, `sage.libs.ecl`)... or `sage-calculus`(?)
 - #29845: PEP 517 `buildapi` for `sage_setup`
 - #29874: Mock integer class
 - #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic

[mkoeppe]

Description changed:

--- 
+++ 
@@ -64,11 +64,12 @@
 - #21559: Install `src/bin` scripts by sagelib's `setup.py`, not make
 - #29850: Install `sage-env-config` as part of `sage_conf`
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
+- #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
 - #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts"
-- #29950: Build sagelib from build/pkgs/sagelib/src, fix `setup.py sdist`, add `spkg-src` and `tox.ini` (supersedes: #21516: Fix sagelib sdist (src/setup.py sdist))
+- #29950: Build sagelib from build/pkgs/sagelib/src, fix `setup.py sdist`, add `spkg-src` and `tox.ini` (supersedes: #21516: Fix sagelib sdist (src/setup.py sdist), #13190: make `sagelib` use `setuptools` instead of `distutils`)
 - #29865: Modularization of sagelib: Break out separate experimental packages `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
 - #29869, #29873, #29892, #29883, #16351, #29881, #29880, #29916, #29922: Improve/repair filtered structure of sagelib.
-- #13190: make `sagelib` use `setuptools` instead of `distutils`
+ 

 **Tickets for Sage 9.3:**

[mkoeppe]

Description changed:

--- 
+++ 
@@ -65,27 +65,37 @@
 - #29850: Install `sage-env-config` as part of `sage_conf`
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
-- #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts"
 - #29950: Build sagelib from build/pkgs/sagelib/src, fix `setup.py sdist`, add `spkg-src` and `tox.ini` (supersedes: #21516: Fix sagelib sdist (src/setup.py sdist), #13190: make `sagelib` use `setuptools` instead of `distutils`)
-- #29865: Modularization of sagelib: Break out separate experimental packages `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
+- #29865: Modularization of sagelib: Prepare experimental subset distributions `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
 - #29869, #29873, #29892, #29883, #16351, #29881, #29880, #29916, #29922: Improve/repair filtered structure of sagelib.

 **Tickets for Sage 9.3:**

-- #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420)
-- Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`, or use distutils, or disable egg building with setuptools using --single-version-externally-managed.
-   - Rework `sage-objects`, `sage-categories`, and `sagelib` as namespace packages.
+- #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420). Keep monolithic package `sagelib` for use in sage-the-distribution. 
+- Deploy mildly modularized distributions `sage-core`, `sage-tdlib`, ... to PyPI.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib`
    - #30151: Modularization of sagelib: Break out a separate package `sage-meataxe`
+- #29039: Deploy `sage_bootstrap` to PyPI
+- #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts". Deploy `sage_setup` to PyPI.
+- #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic
+- #29877: `sage.categories`: Replace import of `ZZ` by `py_scalar_parent(int)`
+
+
+**Tickets for Sage 9.4:**
+
+- #30152: `sage_setup`: Modify `clean_stale_files` to support out-of-tree namespace packages
+- Further modularization:
+   - Rework `sage-objects`, `sage-categories` as namespace packages.
    - #29912: Modularization of sagelib: Break out a separate package `sage-ntl`
    - Modularization of sagelib: Break out a separate package `sage-flint-arb-e_antic`
    - #29941: Modularization of sagelib: Break out a separate package `sage-repl`
    - Modularization of sagelib: Break out a separate package `sage-maxima` (depends on ecl and maxima, packages `sage.interfaces.maxima*`, `sage.libs.ecl`)... or `sage-calculus`(?)
+   - `sage-singular`
+- #29874: Mock integer class
 - #29845: PEP 517 `buildapi` for `sage_setup`
-- #29874: Mock integer class
-- #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic
-- #29877: `sage.categories`: Replace import of `ZZ` by `py_scalar_parent(int)`
+
+

 **Related:**
 - #21508 Meta-ticket: Clean up `src/setup.py` to bring it to standard distutils behavior

[mkoeppe]

Description changed:

--- 
+++ 
@@ -74,7 +74,7 @@

 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420). Keep monolithic package `sagelib` for use in sage-the-distribution. 
 - Deploy mildly modularized distributions `sage-core`, `sage-tdlib`, ... to PyPI.
-   - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib`
+   - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib` (and create `sage-core`)
    - #30151: Modularization of sagelib: Break out a separate package `sage-meataxe`
 - #29039: Deploy `sage_bootstrap` to PyPI
 - #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts". Deploy `sage_setup` to PyPI.

[mkoeppe]

Description changed:

--- 
+++ 
@@ -76,6 +76,7 @@
 - Deploy mildly modularized distributions `sage-core`, `sage-tdlib`, ... to PyPI.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib` (and create `sage-core`)
    - #30151: Modularization of sagelib: Break out a separate package `sage-meataxe`
+   - #29171: `sage-giac` (ex `giacpy_sage` + `sage.libs.giac` + dependency on `libgiac`)
 - #29039: Deploy `sage_bootstrap` to PyPI
 - #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts". Deploy `sage_setup` to PyPI.
 - #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic

[mkoeppe]

Description changed:

--- 
+++ 
@@ -65,6 +65,7 @@
 - #29850: Install `sage-env-config` as part of `sage_conf`
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
+- #29868: pip-installable packages `sage-doc-html`, `sage-doc-pdf`
 - #29950: Build sagelib from build/pkgs/sagelib/src, fix `setup.py sdist`, add `spkg-src` and `tox.ini` (supersedes: #21516: Fix sagelib sdist (src/setup.py sdist), #13190: make `sagelib` use `setuptools` instead of `distutils`)
 - #29865: Modularization of sagelib: Prepare experimental subset distributions `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
 - #29869, #29873, #29892, #29883, #16351, #29881, #29880, #29916, #29922: Improve/repair filtered structure of sagelib.

[mkoeppe]

Description changed:

--- 
+++ 
@@ -78,6 +78,7 @@
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib` (and create `sage-core`)
    - #30151: Modularization of sagelib: Break out a separate package `sage-meataxe`
    - #29171: `sage-giac` (ex `giacpy_sage` + `sage.libs.giac` + dependency on `libgiac`)
+   - #30332: `sage-brial` (`sage.rings.polynomial.pbori`, `sage.libs.polybori`, `sage.crypto.boolean_function`)
 - #29039: Deploy `sage_bootstrap` to PyPI
 - #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts". Deploy `sage_setup` to PyPI.
 - #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic

[mkoeppe]

Description changed:

--- 
+++ 
@@ -64,8 +64,6 @@
 - #21559: Install `src/bin` scripts by sagelib's `setup.py`, not make
 - #29850: Install `sage-env-config` as part of `sage_conf`
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
-- #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
-- #29868: pip-installable packages `sage-doc-html`, `sage-doc-pdf`
 - #29950: Build sagelib from build/pkgs/sagelib/src, fix `setup.py sdist`, add `spkg-src` and `tox.ini` (supersedes: #21516: Fix sagelib sdist (src/setup.py sdist), #13190: make `sagelib` use `setuptools` instead of `distutils`)
 - #29865: Modularization of sagelib: Prepare experimental subset distributions `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
 - #29869, #29873, #29892, #29883, #16351, #29881, #29880, #29916, #29922: Improve/repair filtered structure of sagelib.
@@ -73,6 +71,8 @@

 **Tickets for Sage 9.3:**

+- #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
+- #29868: pip-installable packages `sage-doc-html`, `sage-doc-pdf`
 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420). Keep monolithic package `sagelib` for use in sage-the-distribution. 
 - Deploy mildly modularized distributions `sage-core`, `sage-tdlib`, ... to PyPI.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib` (and create `sage-core`)

[mkoeppe]

Changed commit from 2ccf6b7 to none

[mkoeppe]

Description changed:

--- 
+++ 
@@ -106,7 +106,3 @@

-The branch on this meta-ticket is not intended for merging.
-It is an integration branch to facilitate testing. See https://github.com/mkoeppe/sage/pull/35 for automated tests
-
-

[mkoeppe]

Changed branch from u/mkoeppe/META-modularize-sagelib to none

[mkoeppe]

Description changed:

--- 
+++ 
@@ -62,17 +62,16 @@
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
 - #29803: Upgrade `setuptools`, `setuptools_scm`, `pip` (2020-06), add package `wheel`; remove `zope_interface`
 - #21559: Install `src/bin` scripts by sagelib's `setup.py`, not make
-- #29850: Install `sage-env-config` as part of `sage_conf`
-- Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #29950: Build sagelib from build/pkgs/sagelib/src, fix `setup.py sdist`, add `spkg-src` and `tox.ini` (supersedes: #21516: Fix sagelib sdist (src/setup.py sdist), #13190: make `sagelib` use `setuptools` instead of `distutils`)
-- #29865: Modularization of sagelib: Prepare experimental subset distributions `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
 - #29869, #29873, #29892, #29883, #16351, #29881, #29880, #29916, #29922: Improve/repair filtered structure of sagelib.
- 
+

 **Tickets for Sage 9.3:**

+- #29500 Install all Python packages via `pip wheel`, create PEP 503 simple repository for wheels
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
-- #29868: pip-installable packages `sage-doc-html`, `sage-doc-pdf`
+- #29868: pip-installable package `sage-doc`
+- #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420). Keep monolithic package `sagelib` for use in sage-the-distribution. 
 - Deploy mildly modularized distributions `sage-core`, `sage-tdlib`, ... to PyPI.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib` (and create `sage-core`)
@@ -81,6 +80,7 @@
    - #30332: `sage-brial` (`sage.rings.polynomial.pbori`, `sage.libs.polybori`, `sage.crypto.boolean_function`)
 - #29039: Deploy `sage_bootstrap` to PyPI
 - #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts". Deploy `sage_setup` to PyPI.
+- #29865: Modularization of sagelib: Prepare experimental subset distributions `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
 - #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic
 - #29877: `sage.categories`: Replace import of `ZZ` by `py_scalar_parent(int)`

[mkoeppe]

Description changed:

--- 
+++ 
@@ -64,7 +64,8 @@
 - #21559: Install `src/bin` scripts by sagelib's `setup.py`, not make
 - #29950: Build sagelib from build/pkgs/sagelib/src, fix `setup.py sdist`, add `spkg-src` and `tox.ini` (supersedes: #21516: Fix sagelib sdist (src/setup.py sdist), #13190: make `sagelib` use `setuptools` instead of `distutils`)
 - #29869, #29873, #29892, #29883, #16351, #29881, #29880, #29916, #29922: Improve/repair filtered structure of sagelib.
-
+- #29171: Move `giacpy_sage` into sage source code
+- #30332: Merge `sage_brial` into sagelib

 **Tickets for Sage 9.3:**

@@ -76,8 +77,8 @@
 - Deploy mildly modularized distributions `sage-core`, `sage-tdlib`, ... to PyPI.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib` (and create `sage-core`)
    - #30151: Modularization of sagelib: Break out a separate package `sage-meataxe`
-   - #29171: `sage-giac` (ex `giacpy_sage` + `sage.libs.giac` + dependency on `libgiac`)
-   - #30332: `sage-brial` (`sage.rings.polynomial.pbori`, `sage.libs.polybori`, `sage.crypto.boolean_function`)
+   - `sage-giac` (ex `giacpy_sage` in `sage.libs.giac` + dependency on `libgiac`)
+   - `sage-brial` (ex `sage-brial` in `sage.rings.polynomial.pbori` +  `sage.libs.polybori`, `sage.crypto.boolean_function`)
 - #29039: Deploy `sage_bootstrap` to PyPI
 - #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts". Deploy `sage_setup` to PyPI.
 - #29865: Modularization of sagelib: Prepare experimental subset distributions `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.

[mkoeppe]

Description changed:

--- 
+++ 
@@ -27,7 +27,7 @@

 - Inconsistent handling of symlinks by `pip`/`setuptools`. We cannot simply create a distribution of a subset of the files in `src` by creating a flock of symlinks (see #29850) and expect everything to work; notably, pip fails because of longstanding issues with installs from local directories (see discussion in #29854).

-  Solution: Do not use pip for installations from local source directories.  Use `setup.py install` (distutils) directly; or use `setup.py sdist`, followed by `pip install` of the sdist; or use a custom PEP 517 backend (#29845) for making an sdist, followed by `pip install` of the sdist.
+  Solution: Do not use pip for installations from local source directories.  Use `setup.py install` (distutils) directly; or use `setup.py sdist`, followed by `pip install` of the sdist; or use `setup.py bdist_wheel`, followed by `pip install` of the wheel; or use a custom PEP 517 backend (#29845) for making an sdist, followed by `pip install` of the sdist.

 - The category framework can not really be separated from the `Parent`/`Element`/`CategoryObject` (e.g. coercion is involved in comparisons and binary operations).

[mkoeppe]

Description changed:

--- 
+++ 
@@ -69,7 +69,8 @@

 **Tickets for Sage 9.3:**

-- #29500 Install all Python packages via `pip wheel`, create PEP 503 simple repository for wheels
+- #29500 Install all Python packages via `pip wheel`, store wheels in $SAGE_LOCAL/var/lib/sage/wheels
+- #30527 Create PEP 503 simple repository for wheels built during installation
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
 - #29868: pip-installable package `sage-doc`
 - #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.

[mkoeppe]

Description changed:

--- 
+++ 
@@ -82,6 +82,7 @@
    - `sage-brial` (ex `sage-brial` in `sage.rings.polynomial.pbori` +  `sage.libs.polybori`, `sage.crypto.boolean_function`)
 - #29039: Deploy `sage_bootstrap` to PyPI
 - #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts". Deploy `sage_setup` to PyPI.
+- #30036: pip-installable sage
 - #29865: Modularization of sagelib: Prepare experimental subset distributions `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
 - #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic
 - #29877: `sage.categories`: Replace import of `ZZ` by `py_scalar_parent(int)`

[mkoeppe]

Description changed:

--- 
+++ 
@@ -98,6 +98,7 @@
    - #29941: Modularization of sagelib: Break out a separate package `sage-repl`
    - Modularization of sagelib: Break out a separate package `sage-maxima` (depends on ecl and maxima, packages `sage.interfaces.maxima*`, `sage.libs.ecl`)... or `sage-calculus`(?)
    - `sage-singular`
+- #30534 Repackage `pynac` as a pip-installable package
 - #29874: Mock integer class
 - #29845: PEP 517 `buildapi` for `sage_setup`

[mkoeppe]

Description changed:

--- 
+++ 
@@ -72,6 +72,7 @@
 - #29500 Install all Python packages via `pip wheel`, store wheels in $SAGE_LOCAL/var/lib/sage/wheels
 - #30527 Create PEP 503 simple repository for wheels built during installation
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
+- #22731: Replace `$SAGE_LOCAL/bin` by more specific variables to make Sage easier to package, use in venvs
 - #29868: pip-installable package `sage-doc`
 - #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420). Keep monolithic package `sagelib` for use in sage-the-distribution. 

[mkoeppe]

Description changed:

--- 
+++ 
@@ -84,6 +84,7 @@
 - #29039: Deploy `sage_bootstrap` to PyPI
 - #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts". Deploy `sage_setup` to PyPI.
 - #30036: pip-installable sage
+- #30587: Remove import of `ppl` at startup
 - #29865: Modularization of sagelib: Prepare experimental subset distributions `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
 - #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic
 - #29877: `sage.categories`: Replace import of `ZZ` by `py_scalar_parent(int)`

[mkoeppe]

Description changed:

--- 
+++ 
@@ -72,6 +72,7 @@
 - #29500 Install all Python packages via `pip wheel`, store wheels in $SAGE_LOCAL/var/lib/sage/wheels
 - #30527 Create PEP 503 simple repository for wheels built during installation
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
+- #30606: `sage.features.Feature.resolution`: If `SAGE_ROOT` is available, recommend system packages
 - #22731: Replace `$SAGE_LOCAL/bin` by more specific variables to make Sage easier to package, use in venvs
 - #29868: pip-installable package `sage-doc`
 - #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.

[mkoeppe]

Description changed:

--- 
+++ 
@@ -77,6 +77,7 @@
 - #29868: pip-installable package `sage-doc`
 - #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420). Keep monolithic package `sagelib` for use in sage-the-distribution. 
+- #30666 Add "sage_setup: distribution" directives to all Cython modules needing external libraries
 - Deploy mildly modularized distributions `sage-core`, `sage-tdlib`, ... to PyPI.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib` (and create `sage-core`)
    - #30151: Modularization of sagelib: Break out a separate package `sage-meataxe`

[mkoeppe]

Description changed:

--- 
+++ 
@@ -73,6 +73,8 @@
 - #30527 Create PEP 503 simple repository for wheels built during installation
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
 - #30606: `sage.features.Feature.resolution`: If `SAGE_ROOT` is available, recommend system packages
+- #30647: Make `lazy_import` more friendly to pyflakes and other static checkers
+- #30616: Replace use of `sage.misc.package.PackageNotFoundError` (2); lazy_import: Add keyword argument 'feature'
 - #22731: Replace `$SAGE_LOCAL/bin` by more specific variables to make Sage easier to package, use in venvs
 - #29868: pip-installable package `sage-doc`
 - #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.

[mkoeppe]

Description changed:

--- 
+++ 
@@ -66,10 +66,10 @@
 - #29869, #29873, #29892, #29883, #16351, #29881, #29880, #29916, #29922: Improve/repair filtered structure of sagelib.
 - #29171: Move `giacpy_sage` into sage source code
 - #30332: Merge `sage_brial` into sagelib
+- #29500 Install all Python packages via `pip wheel`, store wheels in $SAGE_LOCAL/var/lib/sage/wheels

 **Tickets for Sage 9.3:**

-- #29500 Install all Python packages via `pip wheel`, store wheels in $SAGE_LOCAL/var/lib/sage/wheels
 - #30527 Create PEP 503 simple repository for wheels built during installation
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
 - #30606: `sage.features.Feature.resolution`: If `SAGE_ROOT` is available, recommend system packages

[mkoeppe]

Description changed:

--- 
+++ 
@@ -71,6 +71,7 @@
 **Tickets for Sage 9.3:**

 - #30527 Create PEP 503 simple repository for wheels built during installation
+- #30715 Remove a few `.all` imports
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
 - #30606: `sage.features.Feature.resolution`: If `SAGE_ROOT` is available, recommend system packages
 - #30647: Make `lazy_import` more friendly to pyflakes and other static checkers

[mkoeppe]

Description changed:

--- 
+++ 
@@ -86,6 +86,7 @@
    - #30151: Modularization of sagelib: Break out a separate package `sage-meataxe`
    - `sage-giac` (ex `giacpy_sage` in `sage.libs.giac` + dependency on `libgiac`)
    - `sage-brial` (ex `sage-brial` in `sage.rings.polynomial.pbori` +  `sage.libs.polybori`, `sage.crypto.boolean_function`)
+   - #30716 Towards making Arb library optional
 - #29039: Deploy `sage_bootstrap` to PyPI
 - #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts". Deploy `sage_setup` to PyPI.
 - #30036: pip-installable sage

[mkoeppe]

Description changed:

--- 
+++ 
@@ -67,15 +67,15 @@
 - #29171: Move `giacpy_sage` into sage source code
 - #30332: Merge `sage_brial` into sagelib
 - #29500 Install all Python packages via `pip wheel`, store wheels in $SAGE_LOCAL/var/lib/sage/wheels
+- #30606: `sage.features.Feature.resolution`: If `SAGE_ROOT` is available, recommend system packages
+- #30616: Replace use of `sage.misc.package.PackageNotFoundError` (2); lazy_import: Add keyword argument 'feature'

 **Tickets for Sage 9.3:**

 - #30527 Create PEP 503 simple repository for wheels built during installation
 - #30715 Remove a few `.all` imports
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
-- #30606: `sage.features.Feature.resolution`: If `SAGE_ROOT` is available, recommend system packages
 - #30647: Make `lazy_import` more friendly to pyflakes and other static checkers
-- #30616: Replace use of `sage.misc.package.PackageNotFoundError` (2); lazy_import: Add keyword argument 'feature'
 - #22731: Replace `$SAGE_LOCAL/bin` by more specific variables to make Sage easier to package, use in venvs
 - #29868: pip-installable package `sage-doc`
 - #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.

[mkoeppe]

Description changed:

--- 
+++ 
@@ -74,16 +74,19 @@

 - #30527 Create PEP 503 simple repository for wheels built during installation
 - #30715 Remove a few `.all` imports
+- #30739 Remove a few more `.all` imports
+- #30779 Duplicate `src/setup.py`
+- #30780 `sage_setup`: Separate installing and cleaning
+- #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420)
+- #30151: Modularization of sagelib: Break out a separate package `sage-meataxe`
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
 - #30647: Make `lazy_import` more friendly to pyflakes and other static checkers
 - #22731: Replace `$SAGE_LOCAL/bin` by more specific variables to make Sage easier to package, use in venvs
 - #29868: pip-installable package `sage-doc`
 - #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
-- #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420). Keep monolithic package `sagelib` for use in sage-the-distribution. 
 - #30666 Add "sage_setup: distribution" directives to all Cython modules needing external libraries
 - Deploy mildly modularized distributions `sage-core`, `sage-tdlib`, ... to PyPI.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib` (and create `sage-core`)
-   - #30151: Modularization of sagelib: Break out a separate package `sage-meataxe`
    - `sage-giac` (ex `giacpy_sage` in `sage.libs.giac` + dependency on `libgiac`)
    - `sage-brial` (ex `sage-brial` in `sage.rings.polynomial.pbori` +  `sage.libs.polybori`, `sage.crypto.boolean_function`)
    - #30716 Towards making Arb library optional

[mkoeppe]

Description changed:

--- 
+++ 
@@ -75,6 +75,7 @@
 - #30527 Create PEP 503 simple repository for wheels built during installation
 - #30715 Remove a few `.all` imports
 - #30739 Remove a few more `.all` imports
+- #30784 `sage.matrix`: Resolve circular imports without using `__init__.py`
 - #30779 Duplicate `src/setup.py`
 - #30780 `sage_setup`: Separate installing and cleaning
 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420)

[mkoeppe]

Changed keywords from sd109 to sd109, sd110

[mkoeppe]

Description changed:

--- 
+++ 
@@ -16,6 +16,7 @@

    With this ticket, the user would instead install a new optional package `sage-tdlib` (which has `tdlib` as a dependency); this installs the `Extension` `sage.graphs.graph_decompositions.tdlib` (as a namespace package).

+   This has been completed in Sage 9.2 (#29701).

 **Not** within the scope of this ticket:

@@ -27,7 +28,7 @@

 - Inconsistent handling of symlinks by `pip`/`setuptools`. We cannot simply create a distribution of a subset of the files in `src` by creating a flock of symlinks (see #29850) and expect everything to work; notably, pip fails because of longstanding issues with installs from local directories (see discussion in #29854).

-  Solution: Do not use pip for installations from local source directories.  Use `setup.py install` (distutils) directly; or use `setup.py sdist`, followed by `pip install` of the sdist; or use `setup.py bdist_wheel`, followed by `pip install` of the wheel; or use a custom PEP 517 backend (#29845) for making an sdist, followed by `pip install` of the sdist.
+  Solution (implemented in Sage 9.2, #29500): Do not use pip for installations from local source directories.  Use `setup.py bdist_wheel`, followed by `pip install` of the wheel.

 - The category framework can not really be separated from the `Parent`/`Element`/`CategoryObject` (e.g. coercion is involved in comparisons and binary operations).

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,6 +1,9 @@
 (From https://groups.google.com/d/msg/sage-devel/M9QTWtln6zU/UHwkrmTKBQAJ)

 Modularize sagelib into separate distutils packages, so that these can be built and run with a much smaller set of libraries. The packages can be made available individually on PyPI. The goal would be to do this **without changing the overall structure of the source tree**.
+
+Presentation video: https://researchseminars.org/talk/SageDays110/11/
+

 1. In particular, create a package `sage-objects` that makes Sage objects, the element/parent framework, categories, the coercion system and the related metaclasses available. This package would, for example, not provide or know about integers, polynomials, or matrices (thus avoiding all the dependencies on the various libraries implementing these). Ideally it would only have a build dependency on Cython and no runtime dependencies. 

[mkoeppe]

Description changed:

--- 
+++ 
@@ -76,35 +76,42 @@

 **Tickets for Sage 9.3:**

-- #30527 Create PEP 503 simple repository for wheels built during installation
 - #30715 Remove a few `.all` imports
 - #30739 Remove a few more `.all` imports
 - #30784 `sage.matrix`: Resolve circular imports without using `__init__.py`
 - #30779 Duplicate `src/setup.py`
 - #30780 `sage_setup`: Separate installing and cleaning
+- #22731: Replace `$SAGE_LOCAL/bin` by more specific variables to make Sage easier to package, use in venvs
+- #30013: src/bin/sage-env: Make sure $SAGE_VENV/bin is at the beginning of the PATH       
+- #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
+- #30527: Create PEP 503 simple repository for wheels built during installation
+- #30578: Add src/requirements.txt and src/setup.cfg [install_requires] for installation of sagelib in a venv
+- #30371: In-place (editable) installs of sagelib in a venv
+- #30896: `./configure --without-python`
+- #29039: Deploy `sage_bootstrap` to PyPI
+- #30036: pip-installable sage
+
+**Tickets for Sage 9.4:**
+
+- #30666 Add "sage_setup: distribution" directives to all Cython modules needing external libraries
+- #30647: Make `lazy_import` more friendly to pyflakes and other static checkers
+- #29868: pip-installable package `sage-doc`
 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420)
 - #30151: Modularization of sagelib: Break out a separate package `sage-meataxe`
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
-- #30647: Make `lazy_import` more friendly to pyflakes and other static checkers
-- #22731: Replace `$SAGE_LOCAL/bin` by more specific variables to make Sage easier to package, use in venvs
-- #29868: pip-installable package `sage-doc`
-- #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
-- #30666 Add "sage_setup: distribution" directives to all Cython modules needing external libraries
 - Deploy mildly modularized distributions `sage-core`, `sage-tdlib`, ... to PyPI.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib` (and create `sage-core`)
    - `sage-giac` (ex `giacpy_sage` in `sage.libs.giac` + dependency on `libgiac`)
    - `sage-brial` (ex `sage-brial` in `sage.rings.polynomial.pbori` +  `sage.libs.polybori`, `sage.crypto.boolean_function`)
    - #30716 Towards making Arb library optional
-- #29039: Deploy `sage_bootstrap` to PyPI
 - #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts". Deploy `sage_setup` to PyPI.
-- #30036: pip-installable sage
 - #30587: Remove import of `ppl` at startup
 - #29865: Modularization of sagelib: Prepare experimental subset distributions `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
 - #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic
 - #29877: `sage.categories`: Replace import of `ZZ` by `py_scalar_parent(int)`

-**Tickets for Sage 9.4:**
+**Tickets for Sage 9.5:**

 - #30152: `sage_setup`: Modify `clean_stale_files` to support out-of-tree namespace packages
 - Further modularization:

[mkoeppe]

Description changed:

--- 
+++ 
@@ -82,14 +82,14 @@
 - #30779 Duplicate `src/setup.py`
 - #30780 `sage_setup`: Separate installing and cleaning
 - #22731: Replace `$SAGE_LOCAL/bin` by more specific variables to make Sage easier to package, use in venvs
-- #30013: src/bin/sage-env: Make sure $SAGE_VENV/bin is at the beginning of the PATH       
+- #30013: `src/bin/sage-env`: Make sure `$SAGE_VENV/bin` is at the beginning of the `PATH`
 - #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #30527: Create PEP 503 simple repository for wheels built during installation
-- #30578: Add src/requirements.txt and src/setup.cfg [install_requires] for installation of sagelib in a venv
+- #30578: Add `src/requirements.txt` and `src/setup.cfg` [install_requires] for installation of sagelib in a venv
 - #30371: In-place (editable) installs of sagelib in a venv
 - #30896: `./configure --without-python`
 - #29039: Deploy `sage_bootstrap` to PyPI
-- #30036: pip-installable sage
+- #30036: Deploy `sagelib` to PyPI

 **Tickets for Sage 9.4:**

[mkoeppe]

Description changed:

--- 
+++ 
@@ -86,6 +86,7 @@
 - #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #30527: Create PEP 503 simple repository for wheels built during installation
 - #30578: Add `src/requirements.txt` and `src/setup.cfg` [install_requires] for installation of sagelib in a venv
+- #30912: sagelib: Update metadata for PyPI deployment
 - #30371: In-place (editable) installs of sagelib in a venv
 - #30896: `./configure --without-python`
 - #29039: Deploy `sage_bootstrap` to PyPI

[mkoeppe]

Description changed:

--- 
+++ 
@@ -85,7 +85,7 @@
 - #30013: `src/bin/sage-env`: Make sure `$SAGE_VENV/bin` is at the beginning of the `PATH`
 - #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #30527: Create PEP 503 simple repository for wheels built during installation
-- #30578: Add `src/requirements.txt` and `src/setup.cfg` [install_requires] for installation of sagelib in a venv
+- #30578: Add `src/requirements.txt` for installation of sagelib in a venv
 - #30912: sagelib: Update metadata for PyPI deployment
 - #30371: In-place (editable) installs of sagelib in a venv
 - #30896: `./configure --without-python`