mkoeppe commented 4 years ago

... in analogy of #29864 (sage-tdlib).

To try: make sagemath_meataxe

(or ./configure --enable-sagemath_meataxe && make build)

This is part of #29705: Meta-ticket: Modularize sagelib into separate distributions (distutils packages)

Depends on #34346

CC: @simon-king-jena @kiwifb @jhpalmieri @dimpase

Component: packages: optional

Keywords: sd111

Work Issues: Rebase, rename "sage-..." -> "sagemath-..."

Branch/Commit: u/mkoeppe/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe @ ebe1dd6

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/30151

mkoeppe commented 4 years ago

Branch: u/mkoeppe/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Commit: b040746

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

`16c60ca`	`src/setup.py: Remove meataxe - now taken care of by package sage_meataxe`
`b040746`	`fixup`

simon-king-jena commented 4 years ago

comment:3

Can you elaborate on what is supposed to be done to use meataxe --- both as a C library, as collection of executables, and as optional Cython extension module?

mkoeppe commented 4 years ago

comment:4

spkg meataxe stays as is. It installs the C library and executables. No dependency on anything from Sage.

spkg sagelib will no longer conditionally build the Cython extension module.

Instead, the new spkg sage_meataxe will build the Cython extension module. It declares both sagelib and meataxe as a dependency.

sage_meataxe, in the implementation of this ticket, does not have a source tarball (note there is no checksums.ini). Instead, it is a "script" package, whose sources lie in build/pkgs/sage_meataxe/src. It has the structure of a Python package (distribution), with setup.py doing the build and installation. The actual Cython sources remain in the src/sage tree (using symlinks). (It is possible to build a pip-installable source distribution tarball as well, using the spkg-src script. Eventually, we will upload it to PyPI.)

simon-king-jena commented 4 years ago

comment:5

Replying to @mkoeppe:

spkg sagelib will no longer conditionally build the Cython extension module.

Instead, the new spkg sage_meataxe will build the Cython extension module. It declares both sagelib and meataxe as a dependency.

So, in order to use it, one has to do make sage_meataxe, right? It would automatically install its dependency meataxe, would build the corresponding Cython extension module, AND would also build the documentation of the Cython extension module? And sage -testall would run the doctests from the corresponding Cython extension module if and only if sage_meataxe is installed, so that there is no need to mark all its tests as "optional"?

If the documentation and test suite would be taken care of, it would indeed be very helpful.

mkoeppe commented 4 years ago

comment:6

Replying to @simon-king-jena:

Replying to @mkoeppe:

spkg sagelib will no longer conditionally build the Cython extension module.

Instead, the new spkg sage_meataxe will build the Cython extension module. It declares both sagelib and meataxe as a dependency.

So, in order to use it, one has to do make sage_meataxe, right? It would automatically install its dependency meataxe, would build the corresponding Cython extension module, AND would also build the documentation of the Cython extension module?

Yes.

And sage -testall would run the doctests from the corresponding Cython extension module if and only if sage_meataxe is installed, so that there is no need to mark all its tests as "optional"?

Yes, I would hope so. I have just created #30778 for this idea.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from b040746 to 1e03fef

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

`b060462`	`sage_setup.find.find_python_sources: Handle native namespace packages, 'nonamespace' files`
`d42aa12`	`is_package_or_namespace_package_dir: New, recognize namespace packages by file 'namespace', monkeypatch into Cython`
`29ca24b`	`Make sage.graphs, sage.matrix, sage.libs, sage.interfaces namespace packages`
`1957acd`	`src/sage_setup/clean.py: Update comment`
`8b37214`	`WIP`
`92f1145`	`sage_install_and_clean: If all_distributions is a list, do not clean files that do not belong to this list`
`44a4456`	`src/sage_setup/command/sage_install.py: Also handle modules in namespace packages`
`706bf59`	`Merge branch 't/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_' into t/30151/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe`
`f096870`	`build/pkgs/sage_meataxe: New`
`1e03fef`	`src/setup.py: Remove meataxe - now taken care of by package sage_meataxe`

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

`ad9402f`	`sage_setup.command.sage_install: Fixup: Remove cleaning code (moved to sage_install_and_clean)`
`88455dc`	`Merge branch 't/30780/sage_setup__separate_installing_and_cleaning' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_`
`aacec42`	`Merge branch 't/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_' into t/30151/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe`

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 1e03fef to aacec42

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from aacec42 to ac10389

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

`800f740`	`Merge branch 't/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_' into t/30151/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe`
`3823197`	`build/pkgs/sage_meataxe: New`
`ac10389`	`build/pkgs/sagelib/src/setup.py: Remove meataxe - now taken care of by package sage_meataxe`

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

12b22d9 fixup

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from ac10389 to 12b22d9

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 12b22d9 to 500635d

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

500635d fixup

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

3d3cc42 build/pkgs/sage_meataxe/spkg-install: Install via bdist_wheel

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 500635d to 3d3cc42

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

`99f3cc7`	`build/pkgs/sage_meataxe/src/setup.py: Set built_distributions`
`a9933b6`	`sage_build_cython: Pass built_distributions to find_extra_files`
`10e2432`	`Merge branch 't/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_' into t/30151/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe`

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 3d3cc42 to 10e2432

mkoeppe commented 4 years ago

comment:14

The installation seems to work, but Sage cannot start because of a circular import related from sage.matrix.args. This needs a bit more work.

mkoeppe commented 4 years ago

Author: Matthias Koeppe

mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -1,5 +1,6 @@
 ... in analogy of #29864 (sage-tdlib).

+To try: `make sage_meataxe`

 This is part of #29705: Meta-ticket: Modularize sagelib into separate distributions (distutils packages)

mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -2,6 +2,9 @@

 To try: `make sage_meataxe`

+(or `./configure --enable-sage_meataxe && make build`)
+
+
 This is part of #29705: Meta-ticket: Modularize sagelib into separate distributions (distutils packages)

simon-king-jena commented 4 years ago

Changed dependencies from #28925 to #28925 #28711

simon-king-jena commented 4 years ago

comment:17

28711 previously depended on this ticket. However, because of giving a talk on p_group_cohomology at the next Sage Days, I am interested in getting p_group_cohomology up and running ASAP.

Therefore, I suggest to make this ticket depend on #28711, rather than the other way around. As soon as the circular import problem is solved, one can modify SAGE_ROOT/build/pkgs/p_group_cohomology/dependencies (same for any other package depending on meataxe, if there is any); I believe this change logically belongs here, because it is THIS ticket that makes the change in the dependencies files possible and necessary.

mkoeppe commented 4 years ago

comment:18

No objection here. This ticket certainly won't make it into Sage 9.2. (Not sure if #28711 can make it - it looks like Volker does not intend to merge non-"blocker" tickets.)

mkoeppe commented 4 years ago

Work Issues: Merge #28711 when ready

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 10e2432 to 202914a

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

`ea0d15a`	`sage.matrix: Resolve circular imports without using __init__.py`
`e4ce720`	`Merge branch 't/30784/sage_matrix__resolve_circular_imports_without_using___init___py' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_`
`202914a`	`Merge branch 't/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_' into t/30151/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe`

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

33e0eca build/pkgs/sage_meataxe: Add metadata

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 202914a to 33e0eca

mkoeppe commented 4 years ago

comment:22

Ready for review

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

`6c408b5`	`build/pkgs/sagelib/src/setup.py: Do not clean files that do not belong to us`
`6aaa958`	`sage_install_and_clean: Move all_distributions into class`
`98abe7e`	`Merge branch 't/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_' into t/30151/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe`

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 33e0eca to 98abe7e

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

`39be639`	`FileDocTestSource.in_lib: Handle namespace packages`
`fb56378`	`Merge branch 't/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_' into t/30151/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe`

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 98abe7e to fb56378

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from fb56378 to ebd5ae7

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

`64c5120`	`find_python_sources: Also collect namespace packages - so that package_data works`
`ebd5ae7`	`Merge branch 't/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_' into t/30151/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe`

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Changed commit from ebd5ae7 to 6cbcbec

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

`4f5a810`	`Upgrade p_group_cohomology to version 3.3.2`
`de8adfa`	`Revert a change to "dependencies" of p_group_cohomology`
`cf60884`	`Add upstream_url to p_group_cohomology`
`ae50157`	`p_group_cohomology's upstream changed by some new commits`
`bd34c47`	`build/pkgs/p_group_cohomology/checksums.ini: Fix upstream_url`
`245560d`	`Fix docbuild of p_group_cohomology`
`f4e5f6a`	`Fix a doctest in sage/tests/modular_cohomology.py`
`649ba8d`	`Merge branch 'u/SimonKing/fix_spkg_install_of_p_group_cohomology' of git://trac.sagemath.org/sage into t/30151/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe`
`6cbcbec`	`build/pkgs/sage_meataxe: Move dependencies here from build/pkgs/p_group_cohomology/dependencies, add sage_meataxe there`

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

b984eeb build/pkgs/p_group_cohomology/spkg-install.in: Remove check for sage-meataxe modules

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Changed commit from 6cbcbec to b984eeb

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

`3c7d675`	`src/sage_setup/find.py (find_python_sources): Update doc, doctests`
`11637a6`	`Merge branch 't/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_' into t/30151/modularization_of_sagelib__break_out_a_separate_package_sage_meataxe`

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Changed commit from b984eeb to 11637a6

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Changed commit from 11637a6 to 7cedb3d

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

7cedb3d build/pkgs/sagelib/src/setup.py: Fix up all_distributions

mkoeppe commented 3 years ago

Changed keywords from none to sd111

mkoeppe commented 3 years ago

Changed work issues from Merge #28711 when ready to Rebase

sagemath / sage

Modularization of sagelib: Break out a separate package sagemath-meataxe #30151

28711 previously depended on this ticket. However, because of giving a talk on p_group_cohomology at the next Sage Days, I am interested in getting p_group_cohomology up and running ASAP.