search

sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.44k stars 480 forks source link

p_group_cohomology does not build on Python3 #28414

Closed dimpase closed 5 years ago

dimpase commented 5 years ago

The aim of this ticket is to provide a new version of the optional p_group_cohomology spkg so that it builds and passes tests both with python-2 and python-3.

Hopefully the version number 3.3 (two times three) is appropriate.

Package documentation

Source ball

p_group_cohomology at Travis CI

Depends on #28444

CC: @simon-king-jena

Component: packages: optional

Author: Simon King

Branch: 6c5dafc

Reviewer: John Palmieri

Issue created by migration from https://trac.sagemath.org/ticket/28414

dimpase commented 5 years ago
comment:100

So, would this still pop up if right before running these tests you removed ~/.sage/ ?

simon-king-jena commented 5 years ago
comment:101

Replying to @dimpase:

So, would this still pop up if right before running these tests you removed ~/.sage/ ?

See comment:94. There, it didn't help. Now, it doesn't either:

king@klap:~/Sage/git$ mv ~/.sage/ ~/.sageBAK
king@klap:~/Sage/git$ mkdir ~/.sage
king@klap:~/Sage/git$ sage/sage 
┌────────────────────────────────────────────────────────────────────┐
│ SageMath version 8.9.beta8, Release Date: 2019-08-25               │
│ Using Python 2.7.15. Type "help()" for help.                       │
└────────────────────────────────────────────────────────────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Warning: this is a prerelease version, and it may be unstable.     ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
Setting permissions of DOT_SAGE directory so only you can read and write it.
sage: from pGroupCohomology import CohomologyRing
sage: from pGroupCohomology.cohomology import COHO
sage: CohomologyRing.doctest_setup()
sage: CohomologyRing.global_options('debug')
sage: _ = COHO()
Critical bug in python.
kwds.get('root', COHO.workspace) is <type 'str'>
COHO.workspace is <type 'str'>
str=<type 'unicode'>, unicode=<type 'unicode'>
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-5-a303ce041f27> in <module>()
----> 1 _ = COHO()

pGroupCohomology/cohomology.pyx in pGroupCohomology.cohomology.COHO.__init__()

RuntimeError: An explicit cast to <str> resulted in a unicode object
simon-king-jena commented 5 years ago
comment:102

I wonder if the problem occurs because I am not installing the development versions of my package from an actual "package" (using sage -i p_group_cohomology), but directly from the sources. I.e., in the src folder of my development tree, I do

sage -python setup.py build
sage -pip install . --upgrade

I know that sage -python setup.py build does create .c files in the src folder, but since the changed module is recompiled both in py2 and py3, I thought that it doesn't matter. And for efficiency, I want that only modules are recompiled that have changed.

So, I would like to test if the problem persists when separating the py2- and py3 build folders. When doing sage -python setup.py build, is it somehow possible to prescribe in what folder the .c and .pyc files are created? Then, I could choose separate folders for py2 and py3.

simon-king-jena commented 5 years ago
comment:103

I am now putting cythonize(..., build_dir=os.path.join("build","c_files-{}.{}".format(version_info.major, version_info.minor))) into my setup.py. In that way, the c-files will be created in separate directories for both Sage versions.

Let's see if this is enough to solve the problem!

simon-king-jena commented 5 years ago
comment:104

PS: Even if it solves the problem, I will test upon import whether str is unicode in py2, and if it is, raise a RuntimeError advising to recompile. I hope this error will never pop up...

simon-king-jena commented 5 years ago
comment:105

After changing setup.py so that the .c files will be in different folders for different python versions, tests in cohomology.pyx pass both with py2 and py3 and then py2 again. So, I can now turn to modular_cohomology.pyx and finally init.py (which contains a long doc string, namely the "general overview" of the package).

simon-king-jena commented 5 years ago
comment:106

Hooray! When I ran sage -tp 2 --force-lib --long --warn-long 60.1, all tests passed both in python 2 and python 3. I don't know how significant the run time differences are. In any case, here are the details:

Python-3
========
Running doctests with ID 2019-09-25-00-50-30-78efa273.
Using --optional=build,dochtml,gap_packages,libsemigroups,meataxe,memlimit,mpir,python2,sage
Sorting sources by runtime so that slower doctests are run first....
Doctesting 13 files using 2 threads.
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/cochain.pxd
    [0 tests, 0.00 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/cochain.pyx
    [1481 tests, 350.33 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/cohomology.pyx
    [1445 tests, 363.25 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/isomorphism_test.py
**********************************************************************
File "/home/king/Projekte/coho/coho-devel/src/pGroupCohomology/isomorphism_test.py", line 67, in pGroupCohomology.isomorphism_test.IsomorphismTest
Warning, slow doctest:
    T.explore_isomorphisms()        # long time
Test ran for 122.79 s
    [213 tests, 221.05 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/modular_cohomology.pyx
    [459 tests, 274.13 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/factory.py
    [229 tests, 126.99 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/__init__.py
    [343 tests, 96.76 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/dickson.pyx
    [28 tests, 1.53 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/auxiliaries.py
    [39 tests, 0.79 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/barcode.py
    [143 tests, 10.31 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/resolution.pxd
    [0 tests, 0.00 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/resolution_bindings.pxd
    [0 tests, 0.00 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/resolution.pyx
    [882 tests, 22.95 s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: 765.7 seconds
    cpu time: 939.6 seconds
    cumulative wall time: 1468.1 seconds

Python-2
========
Running doctests with ID 2019-09-25-01-04-47-fa1de7f4.
Git branch: matrix_implementation_keyword
Using --optional=build,ccache,dochtml,frobby,gap_packages,gdb,gfortran,libsemigroups,meataxe,memlimit,mpir,p_group_cohomology,python2,sage
Sorting sources by runtime so that slower doctests are run first....
Doctesting 13 files using 2 threads.
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/cochain.pyx
    [1481 tests, 437.44 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/cochain.pxd
    [0 tests, 0.00 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/cohomology.pyx
    [1445 tests, 465.15 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/isomorphism_test.py
**********************************************************************
File "/home/king/Projekte/coho/coho-devel/src/pGroupCohomology/isomorphism_test.py", line 67, in pGroupCohomology.isomorphism_test.IsomorphismTest
Warning, slow doctest:
    T.explore_isomorphisms()        # long time
Test ran for 116.79 s
    [213 tests, 229.85 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/modular_cohomology.pyx
    [459 tests, 312.40 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/factory.py
    [229 tests, 124.88 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/__init__.py
    [343 tests, 96.86 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/resolution.pxd
    [0 tests, 0.00 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/resolution.pyx
    [882 tests, 21.38 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/dickson.pyx
    [28 tests, 1.63 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/auxiliaries.py
    [39 tests, 0.78 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/resolution_bindings.pxd
    [0 tests, 0.00 s]
sage -t --long --warn-long 60.1 /home/king/Projekte/coho/coho-devel/src/pGroupCohomology/barcode.py
    [143 tests, 14.90 s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: 873.5 seconds
    cpu time: 996.8 seconds
    cumulative wall time: 1705.3 seconds

Hence, tomorrow (i.e., in central European time, today in a few hours :-)) I will create a new package version and post it here for review.

simon-king-jena commented 5 years ago
comment:107

Something more to do: I think I should add a test showing what happens when moving the directory associated with a cohomology ring. The package can cope with moved data, based on some trickery in the attribute _default_filename that is defined by Sage's unpickler. I've turned _default_filename into a property (with getter, setter and deleter), and its documentation should contain the new test.

simon-king-jena commented 5 years ago
comment:108

I'm afraid that another test failure popped up. So, I cannot post the new version yet.

simon-king-jena commented 5 years ago

Description changed:

--- 
+++ 
@@ -1,21 +1,3 @@
+The aim of this ticket is to provide a new version of the optional `p_group_cohomology` spkg so that it builds and passes tests both with python-2 and python-3.

-```
-[p_group_cohomology-3.2] Installing p_group_cohomology-3.2
-[p_group_cohomology-3.2] 
-[p_group_cohomology-3.2] Error compiling Cython file:
-[p_group_cohomology-3.2] ------------------------------------------------------------
-[p_group_cohomology-3.2] ...
-[p_group_cohomology-3.2]     ## If it doesn't, then we need to find out *after* loading
-[p_group_cohomology-3.2]     second_cache_attempt = False
-[p_group_cohomology-3.2]     if not coho_options.has_key('@newroot@'):
-[p_group_cohomology-3.2]         try:
-[p_group_cohomology-3.2]             OUT = _cache.get((GroupKey,StateFile)) or _cache.get((GroupKey,StateFile[:-5]))
-[p_group_cohomology-3.2]         except StandardError, msg:
-[p_group_cohomology-3.2]               ^
-[p_group_cohomology-3.2] ------------------------------------------------------------
-[p_group_cohomology-3.2] 
-[p_group_cohomology-3.2] pGroupCohomology/cohomology.pyx:160:15: undeclared name not builtin: StandardError
-[p_group_cohomology-3.2] 
-```
-`StandardError` has been removed in Python3.
-Cf. e.g. http://python3porting.com/differences.html#standarderror
+Hopefully the version number 3.3 (two times three) is appropriate.
simon-king-jena commented 5 years ago
comment:109

After fixing the failed test (in which I forgot to save something before loading it), I added a new feature to unit_test_64: Previously it was only tested whether the generator degrees and the Poincaré series of a freshly computed cohomology ring is consistent with what is stored in the database; now I test whether the new ring and the stored ring are isomorphic.

Indeed they are, for all groups of order 64. More precisely: In many cases the generators of the relation ideal are not identical. But in all cases, mapping the i-th generator of the new ring to the i-th generator of the old ring results in an isomorphism.

There is one remaining py3 issue, though: When I try to build the documentation, I get the error "No module named sphinx". The documentation does build, when I use py2. What can I do to fix it in py3?

EDIT: Is it not the case that sphinx is installed in Sage for building its documentation, both in py2 and py3? I am not sure if I ever tried to build the documentation of Sage-with-py3, though. Will sphinx be installed when I do a plain make in Sage (i.e., when I build the docs)?

jhpalmieri commented 5 years ago
comment:110

Sphinx should be installed, yes. Look for SAGE_ROOT/local/lib/python3.7/site-packages/sphinx/. If you run Sage, can you import sphinx?

Edit: this is for a Python 3 build of Sage.

simon-king-jena commented 5 years ago
comment:111

Replying to @jhpalmieri:

Sphinx should be installed, yes. Look for SAGE_ROOT/local/lib/python3.7/site-packages/sphinx/. If you run Sage, can you import sphinx?

It was a trivial mistake: In the doc's Makefile, I had SPHINXBUILD = python -msphinx, but since python means python-2 even in a py3-install of Sage, I have to use SPHINXBUILD = sage -python -msphinx

simon-king-jena commented 5 years ago

Branch: u/SimonKing/p_group_cohomology_in_py_3

simon-king-jena commented 5 years ago
comment:113

I checked that the package builds and passes tests in Sage-with-py3. I checked that when running the tests directly on the sources (not during package installation) the tests pass in Sage-with-py2. I recommend that whoever reviews this tries the package both in py2 and in py3.

New commits:

2e0d11ePass unpickling options to pickle.load, default encoding 'latin1'. Accept both str and bytes in mtx_unpickle
8c105cdMake str_to_bytes/bytes_to_str accept both str and bytes input.
c8a0748Add tests for #28444
89ac77aFix keyword for py3-only test
3693024Fix doc strings in sage.misc.persist
f0828eeAdd a comment regarding the expected data type for an unpickle helper
ba41ebeFix two typos in a comment
463806cUpgrade p_group_cohomology to version 3.3
simon-king-jena commented 5 years ago

Commit: 463806c

simon-king-jena commented 5 years ago

Author: Simon King

simon-king-jena commented 5 years ago

Description changed:

--- 
+++ 
@@ -1,3 +1,7 @@
 The aim of this ticket is to provide a new version of the optional `p_group_cohomology` spkg so that it builds and passes tests both with python-2 and python-3.

 Hopefully the version number 3.3 (two times three) is appropriate.
+
+[Package documentation](https://users.fmi.uni-jena.de/cohomology/documentation/)
+
+[Source ball](https://users.fmi.uni-jena.de/cohomology/p_group_cohomology-3.3.tar.xz)
simon-king-jena commented 5 years ago
comment:114

Hooray! In Py2, sage -f -c p_group_cohomology results in

...
[p_group_cohomology-3.3] Running the test suite for p_group_cohomology-3.3...
[p_group_cohomology-3.3] make[2]: Entering directory '/home/king/Sage/git/sage/local/var/tmp/sage/build/p_group_cohomology-3.3/src/modular_resolution-1.1'
[p_group_cohomology-3.3] Making check in src
[p_group_cohomology-3.3] make[3]: Entering directory '/home/king/Sage/git/sage/local/var/tmp/sage/build/p_group_cohomology-3.3/src/modular_resolution-1.1/src'
[p_group_cohomology-3.3] make  mnttest.sh
[p_group_cohomology-3.3] make[4]: Entering directory '/home/king/Sage/git/sage/local/var/tmp/sage/build/p_group_cohomology-3.3/src/modular_resolution-1.1/src'
[p_group_cohomology-3.3] test -r ./test.reg || cp ./test.reg .
[p_group_cohomology-3.3] make[4]: Leaving directory '/home/king/Sage/git/sage/local/var/tmp/sage/build/p_group_cohomology-3.3/src/modular_resolution-1.1/src'
[p_group_cohomology-3.3] make  check-TESTS
[p_group_cohomology-3.3] make[4]: Entering directory '/home/king/Sage/git/sage/local/var/tmp/sage/build/p_group_cohomology-3.3/src/modular_resolution-1.1/src'
[p_group_cohomology-3.3] make[5]: Entering directory '/home/king/Sage/git/sage/local/var/tmp/sage/build/p_group_cohomology-3.3/src/modular_resolution-1.1/src'
[p_group_cohomology-3.3] PASS: mnttest.sh
[p_group_cohomology-3.3] ============================================================================
[p_group_cohomology-3.3] Testsuite summary for modular_resolution 1.1
[p_group_cohomology-3.3] ============================================================================
[p_group_cohomology-3.3] # TOTAL: 1
[p_group_cohomology-3.3] # PASS:  1
[p_group_cohomology-3.3] # SKIP:  0
[p_group_cohomology-3.3] # XFAIL: 0
[p_group_cohomology-3.3] # FAIL:  0
[p_group_cohomology-3.3] # XPASS: 0
[p_group_cohomology-3.3] # ERROR: 0
[p_group_cohomology-3.3] ============================================================================
[p_group_cohomology-3.3] make[5]: Leaving directory '/home/king/Sage/git/sage/local/var/tmp/sage/build/p_group_cohomology-3.3/src/modular_resolution-1.1/src'
[p_group_cohomology-3.3] make[4]: Leaving directory '/home/king/Sage/git/sage/local/var/tmp/sage/build/p_group_cohomology-3.3/src/modular_resolution-1.1/src'
[p_group_cohomology-3.3] make[3]: Leaving directory '/home/king/Sage/git/sage/local/var/tmp/sage/build/p_group_cohomology-3.3/src/modular_resolution-1.1/src'
[p_group_cohomology-3.3] make[3]: Entering directory '/home/king/Sage/git/sage/local/var/tmp/sage/build/p_group_cohomology-3.3/src/modular_resolution-1.1'
[p_group_cohomology-3.3] make[3]: Leaving directory '/home/king/Sage/git/sage/local/var/tmp/sage/build/p_group_cohomology-3.3/src/modular_resolution-1.1'
[p_group_cohomology-3.3] make[2]: Leaving directory '/home/king/Sage/git/sage/local/var/tmp/sage/build/p_group_cohomology-3.3/src/modular_resolution-1.1'
[p_group_cohomology-3.3] Running doctests with ID 2019-09-26-11-19-11-b01ece2b.
[p_group_cohomology-3.3] Git branch: t/28414/p_group_cohomology_in_py_3
[p_group_cohomology-3.3] Using --optional=build,ccache,dochtml,frobby,gap_packages,gdb,gfortran,libsemigroups,meataxe,memlimit,mpir,python2,sage
[p_group_cohomology-3.3] Sorting sources by runtime so that slower doctests are run first....
[p_group_cohomology-3.3] Doctesting 13 files using 3 threads.
[p_group_cohomology-3.3] sage -t --long --warn-long 80.0 pGroupCohomology-3.3/pGroupCohomology/modular_cohomology.pyx
[p_group_cohomology-3.3]     [459 tests, 456.02 s]
[p_group_cohomology-3.3] sage -t --long --warn-long 80.0 pGroupCohomology-3.3/pGroupCohomology/cochain.pxd
[p_group_cohomology-3.3]     [0 tests, 0.00 s]
[p_group_cohomology-3.3] sage -t --long --warn-long 80.0 pGroupCohomology-3.3/pGroupCohomology/cochain.pyx
[p_group_cohomology-3.3]     [1481 tests, 599.35 s]
[p_group_cohomology-3.3] sage -t --long --warn-long 80.0 pGroupCohomology-3.3/pGroupCohomology/cohomology.pyx
[p_group_cohomology-3.3]     [1465 tests, 606.86 s]
[p_group_cohomology-3.3] sage -t --long --warn-long 80.0 pGroupCohomology-3.3/pGroupCohomology/isomorphism_test.py
[p_group_cohomology-3.3] **********************************************************************
[p_group_cohomology-3.3] File "pGroupCohomology-3.3/pGroupCohomology/isomorphism_test.py", line 67, in pGroupCohomology.isomorphism_test.IsomorphismTest
[p_group_cohomology-3.3] Warning, slow doctest:
[p_group_cohomology-3.3]     T.explore_isomorphisms()        # long time
[p_group_cohomology-3.3] Test ran for 161.47 s
[p_group_cohomology-3.3]     [213 tests, 307.72 s]
[p_group_cohomology-3.3] sage -t --long --warn-long 80.0 pGroupCohomology-3.3/pGroupCohomology/__init__.py
[p_group_cohomology-3.3]     [343 tests, 148.33 s]
[p_group_cohomology-3.3] sage -t --long --warn-long 80.0 pGroupCohomology-3.3/pGroupCohomology/factory.py
[p_group_cohomology-3.3] **********************************************************************
[p_group_cohomology-3.3] File "pGroupCohomology-3.3/pGroupCohomology/factory.py", line 133, in pGroupCohomology.factory.unit_test_64
[p_group_cohomology-3.3] Warning, slow doctest:
[p_group_cohomology-3.3]     L, t = unit_test_64(from_scratch=False)    # long time
[p_group_cohomology-3.3] Test ran for 99.46 s
[p_group_cohomology-3.3]     [229 tests, 209.64 s]
[p_group_cohomology-3.3] sage -t --long --warn-long 80.0 pGroupCohomology-3.3/pGroupCohomology/resolution.pxd
[p_group_cohomology-3.3]     [0 tests, 0.00 s]
[p_group_cohomology-3.3] sage -t --long --warn-long 80.0 pGroupCohomology-3.3/pGroupCohomology/resolution.pyx
[p_group_cohomology-3.3]     [882 tests, 26.85 s]
[p_group_cohomology-3.3] sage -t --long --warn-long 80.0 pGroupCohomology-3.3/pGroupCohomology/dickson.pyx
[p_group_cohomology-3.3]     [28 tests, 1.41 s]
[p_group_cohomology-3.3] sage -t --long --warn-long 80.0 pGroupCohomology-3.3/pGroupCohomology/barcode.py
[p_group_cohomology-3.3]     [143 tests, 21.04 s]
[p_group_cohomology-3.3] sage -t --long --warn-long 80.0 pGroupCohomology-3.3/pGroupCohomology/resolution_bindings.pxd
[p_group_cohomology-3.3]     [0 tests, 0.00 s]
[p_group_cohomology-3.3] sage -t --long --warn-long 80.0 pGroupCohomology-3.3/pGroupCohomology/auxiliaries.py
[p_group_cohomology-3.3]     [39 tests, 0.92 s]
[p_group_cohomology-3.3] ----------------------------------------------------------------------
[p_group_cohomology-3.3] All tests passed!
[p_group_cohomology-3.3] ----------------------------------------------------------------------
[p_group_cohomology-3.3] Total time for all tests: 815.2 seconds
[p_group_cohomology-3.3]     cpu time: 1261.3 seconds
[p_group_cohomology-3.3]     cumulative wall time: 2378.1 seconds
[p_group_cohomology-3.3] 
[p_group_cohomology-3.3] real   13m52.051s
[p_group_cohomology-3.3] user   22m48.984s
[p_group_cohomology-3.3] sys    4m1.098s
[p_group_cohomology-3.3] Successfully installed p_group_cohomology-3.3
[p_group_cohomology-3.3] Deleting temporary build directory
[p_group_cohomology-3.3] /home/king/Sage/git/sage/local/var/tmp/sage/build/p_group_cohomology-3.3
[p_group_cohomology-3.3] Finished installing p_group_cohomology-3.3.spkg
make[1]: Leaving directory '/home/king/Sage/git/sage/build/make'

real    27m19.468s
user    30m7.398s
sys 4m10.930s
Sage build/upgrade complete!

In Py3, I had more warnings about slow tests (six instead of two, if I recall correctly), but perhaps my laptop (or its harddisk) was busy with other things.

Now I am running unit_test_64 in Python-2. Note that in Python-3 I got about 20 (or even more) examples where the freshly computed relation ideal was presented by a different set of generators than what was stored in the database. But, as I have mentioned in comment:82, this seems to be Singular not doing all tail reduction in py2.

simon-king-jena commented 5 years ago
comment:115

Replying to @simon-king-jena:

Now I am running unit_test_64 in Python-2. Note that in Python-3 I got about 20 (or even more) examples where the freshly computed relation ideal was presented by a different set of generators than what was stored in the database.

Just for the record: In the py2 version, the freshly computed results were identical (not only equal up to generator choice) to the stored results.

simon-king-jena commented 5 years ago
comment:116

By the way, at the recent Sage Days in Bonn, I learnt how to host the package repository at github and use Travis CI to some extent. I have posted the link to the ticket description.

Note, however, that testing it on Travis CI with python-3 will be difficult: I was told that the current Sage-with-py3 docker image is rather outdated, and also I don't know how I should write my .travis.yml so that the automatic tests involve both py2 and py3. The answer on sage-devel didn't help me much. Is someone reading this who can tell me how I should modify .travis.yml?

simon-king-jena commented 5 years ago

Description changed:

--- 
+++ 
@@ -5,3 +5,5 @@
 [Package documentation](https://users.fmi.uni-jena.de/cohomology/documentation/)

 [Source ball](https://users.fmi.uni-jena.de/cohomology/p_group_cohomology-3.3.tar.xz)
+
+[p_group_cohomology at Travis CI](https://travis-ci.org/sagemath/p_group_cohomology)
jhpalmieri commented 5 years ago
comment:117

Suppose I delete .sage/pGroupCohomology/, to force the package to reinstall those files. Then

[p_group_cohomology-3.3]     IOError: Saved data at /Users/jpalmier/.sage/pGroupCohomology/db/4gp2/H4gp2.sobj are not readable: expected bytes, unicode found

I guess I have a few questions:

This particular problem should not happen much: people won't be switching from Python 3 to Python 2 very much. So don't work too hard to get the Python 3 version to produce .sobj files which are readable by Python 2.

simon-king-jena commented 5 years ago
comment:118

Replying to @jhpalmieri:

Suppose I delete .sage/pGroupCohomology/, to force the package to reinstall those files. Then

  • if I build with Python 2 first and then Python 3, all tests pass.
  • if I build with Python 3 first and then Python 2, I get failures:
[p_group_cohomology-3.3]     IOError: Saved data at /Users/jpalmier/.sage/pGroupCohomology/db/4gp2/H4gp2.sobj are not readable: expected bytes, unicode found

It was a good idea to do such tests. And I am surprised about the error message: There is no use of bytes in the whole cohomology package. Perhaps the error is about unpickling a MeatAxe matrix (sage.matrix.matrix_gfpn_dense.Matrix_gfpn_dense)? But this was supposed to be fixed in #28444.

I guess I have a few questions:

  • is it possible (and not too much work to implement) for the Python 3 version to build .sobj files which are readable by Python 2?

I thought they were.

  • if not, should the package discard the existing .sobj file and rebuild in this case?
  • more generally, if there is any error reading the .sobj file — maybe somehow the file got corrupted — should it just rebuild?

It is already the case that if some data source provides data in legacy format, it will automatically be overwritten by a new pickle.

This particular problem should not happen much: people won't be switching from Python 3 to Python 2 very much. So don't work too hard to get the Python 3 version to produce .sobj files which are readable by Python 2.

But it should still work and needs to be investigated.

I'll follow your instructions (remove .sage/pGroupCOhomology, install (and also test?) with Python 3 and then install with Python 2 and see what is happening.

jhpalmieri commented 5 years ago
comment:119

Replying to @simon-king-jena:

I'll follow your instructions (remove .sage/pGroupCOhomology, install (and also test?) with Python 3 and then install with Python 2 and see what is happening.

I installed and tested when I did this, but just now I hit the same error with:

sage: from pGroupCohomology import CohomologyRing
sage: CohomologyRing(16, 2)
H^*(SmallGroup(16,2); GF(2))
sage: from pGroupCohomology import CohomologyRing
sage: CohomologyRing(16, 2)
---------------------------------------------------------------------------
IOError                                   Traceback (most recent call last)
<ipython-input-2-e1924d996e12> in <module>()
----> 1 CohomologyRing(Integer(16), Integer(2))

/Users/jpalmier/Desktop/Sage/sage_builds/py2/sage-8.9.rc1/local/lib/python2.7/site-packages/pGroupCohomology/factory.pyc in __call__(self, *args, **kwds)
   1778             if q < 128:
   1779                 extras['websource'] = False
-> 1780             OUT = self._check_compatibility(CacheKey, self._get_p_group_from_cache_or_db(GStem, KEY, **extras) or self._get_p_group_from_scratch(KEY, q, GStem, GroupName, **extras))
   1781             return OUT
   1782 

/Users/jpalmier/Desktop/Sage/sage_builds/py2/sage-8.9.rc1/local/lib/python2.7/site-packages/pGroupCohomology/factory.pyc in _get_p_group_from_cache_or_db(self, GStem, KEY, **kwds)
   1209                 if '@use_this_root@' in coho_options:
   1210                     del coho_options['@use_this_root@']
-> 1211                 raise IOError("Saved data at %s are not readable: %s"%(os.path.join(root_workspace,file_name), msg))
   1212         ## 3. Link with local sources and load from there
   1213         elif root_local_sources and os.access(os.path.join(root_local_sources,file_name), os.R_OK) and not from_scratch:

IOError: Saved data at /Users/jpalmier/.sage/pGroupCohomology/db/16gp2/H16gp2.sobj are not readable: NEWOBJ class argument isn't a type object
simon-king-jena commented 5 years ago
comment:120

Probably the following would work, too (so that it is not always needed to reinstall):

Anyway, after doing the py3 test suit, I can confirm what you reported: The stuff in .sage/pGroupCohomology/db/16gp2 cannot be unpickled with py2. In particular, the state descriptor cannot be loaded:

sage: S = load('/home/king/.sage/pGroupCohomology/db/16gp2/dat/State.sobj')
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-2796f9b01dc5> in <module>()
----> 1 S = load('/home/king/.sage/pGroupCohomology/db/16gp2/dat/State.sobj')

/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.load (build/cythonized/sage/misc/persist.c:2845)()
    157 
    158     ## Load file by absolute filename
--> 159     with open(filename, 'rb') as fobj:
    160         X = loads(fobj.read(), compress=compress, **kwargs)
    161     try:

/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.load (build/cythonized/sage/misc/persist.c:2795)()
    158     ## Load file by absolute filename
    159     with open(filename, 'rb') as fobj:
--> 160         X = loads(fobj.read(), compress=compress, **kwargs)
    161     try:
    162         X._default_filename = os.path.abspath(filename)

/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.loads (build/cythonized/sage/misc/persist.c:6929)()
   1043 
   1044     unpickler = SageUnpickler(io.BytesIO(s), **kwargs)
-> 1045     return unpickler.load()
   1046 
   1047 

/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/misc/persist.pyx in sage.misc.persist._BaseUnpickler.load (build/cythonized/sage/misc/persist.c:5806)()
    655 
    656         def load(self):
--> 657             return self._unpickler.load()
    658 
    659         def noload(self):

/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/structure/factory.pyx in sage.structure.factory.lookup_global (build/cythonized/sage/structure/factory.c:4615)()
    748         Integer Ring
    749     """
--> 750     name = bytes_to_str(name, encoding='ASCII')
    751     try:
    752         return factory_unpickles[name]

/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/cpython/string.pxd in sage.cpython.string.bytes_to_str (build/cythonized/sage/structure/factory.c:5038)()
     55         return b
     56     if type(b) is not bytes:
---> 57         raise TypeError(f"expected bytes, {type(b).__name__} found")
     58 
     59     if PY_MAJOR_VERSION <= 2:

TypeError: expected bytes, unicode found

And I guess this can be fixed in bytes_to_str: After the changes in #28444, bytes_to_str is more tolerant about the input type (it may be str), but it should be even more so. It should (in py2) accept unicode.

dimpase commented 5 years ago
comment:121

I think it's good enough if py2-created pickles work with py3.

Require the opposite compatibility seems to be an overkill.

simon-king-jena commented 5 years ago
comment:122

Replying to @dimpase:

I think it's good enough if py2-created pickles work with py3.

Require the opposite compatibility seems to be an overkill.

I believe the fix (in #28540) would be relatively easy.

If you think that we shouldn't worry so much about reading py3-pickles in py2, then of course the group cohomology package can be reviewed as it is now. If I am right about #28640, then #28540 will fix the unpickling of py3-cohomology in py2 without any change in the package.

nbruin commented 5 years ago
comment:123

Replying to @simon-king-jena:

And I guess this can be fixed in bytes_to_str: After the changes in #28444, bytes_to_str is more tolerant about the input type (it may be str), but it should be even more so. It should (in py2) accept unicode.

I think that at this point it's becoming pretty clear you should use a routine that is not named bytes_to_str, but a routine "ensure_str", which in py3 accepts unicode=str and return the result unchanged, and also accepts py2's "bytes=str" objects and assumes it's "latin1" encoded, because that's what it would have been as a "str" in py2.

You are clearly NOT converting a "bytes" object to a "str" here; you are ensuring something is a "str", doing some conversions if necessary. You should use a routine that reflects that in its name if you need specific behaviour.

The reason why I care is because you are changing routines in sage.cpython.string, which people will expect to contain widely valid; well-defined routines. Stretching them up with hacks for specific purposes will lead to silent erroneous behaviour further down the line, when these routines are used in other contexts because they sound like the right tool (but aren't).

If it gets a py3 "unicode" object in py2, ideally it could just leave it as unicode, because that should be valid in str contexts in py2 for the most time as well. But we may be far from the ideal in py2. A safe choice would be str(<unicode>) to ensure you actually get a py2 str. That uses "ascii" encoding, so it may fail. You could allow for "UTF-8" but that would not round-trip correctly with the "latin1" on the other side. Using "latin1" for encoding unicode-to-str on py2 may also fail, because not all unicode code points have an encoding in "latin1".

simon-king-jena commented 5 years ago
comment:124

Replying to @nbruin:

The reason why I care is because you are changing routines in sage.cpython.string, which people will expect to contain widely valid; well-defined routines. Stretching them up with hacks for specific purposes will lead to silent erroneous behaviour further down the line, when these routines are used in other contexts because they sound like the right tool (but aren't).

Agreed. I have already changed the topic of #28540.

If it gets a py3 "unicode" object in py2, ideally it could just leave it as unicode, because that should be valid in str contexts in py2 for the most time as well. But we may be far from the ideal in py2. A safe choice would be str(<unicode>) to ensure you actually get a py2 str. That uses "ascii" encoding, so it may fail.

The application (and the place where #28540 should be changed) is sage.structure.factory.lookup_global. This function used to only accept str. By #28444, it also accepts bytes, which makes it possible to unpickle a py2-pickle in py3.

I suggest in #28540 that it also accepts a unicode. This makes it possible to unpickle a py3-pickle in py2.

Again, I won't make this a dependency for the cohomology package.

simon-king-jena commented 5 years ago
comment:125

28540 brought an improvement:

sage: from pGroupCohomology import CohomologyRing
sage: H = CohomologyRing(16,2,options='debug')
_get_p_group_from_cache_or_db:
    Data found at /home/king/.sage/pGroupCohomology/db/16gp2/H16gp2.sobj
    Group data are rooted at '/home/king/.sage/pGroupCohomology/db'.
Loading a cohomology ring:
    The state descriptor of the to-be-unpickled ring is expected to be provided at u'/home/king/.sage/pGroupCohomology/db/16gp2/dat/State.sobj'
---------------------------------------------------------------------------
IOError                                   Traceback (most recent call last)
<ipython-input-2-a62afc150e39> in <module>()
----> 1 H = CohomologyRing(Integer(16),Integer(2),options='debug')

/home/king/Sage/git/sage/local/lib/python2.7/site-packages/pGroupCohomology/factory.pyc in __call__(self, *args, **kwds)
   1778             if q < 128:
   1779                 extras['websource'] = False
-> 1780             OUT = self._check_compatibility(CacheKey, self._get_p_group_from_cache_or_db(GStem, KEY, **extras) or self._get_p_group_from_scratch(KEY, q, GStem, GroupName, **extras))
   1781             return OUT
   1782 

/home/king/Sage/git/sage/local/lib/python2.7/site-packages/pGroupCohomology/factory.pyc in _get_p_group_from_cache_or_db(self, GStem, KEY, **kwds)
   1209                 if '@use_this_root@' in coho_options:
   1210                     del coho_options['@use_this_root@']
-> 1211                 raise IOError("Saved data at %s are not readable: %s"%(os.path.join(root_workspace,file_name), msg))
   1212         ## 3. Link with local sources and load from there
   1213         elif root_local_sources and os.access(os.path.join(root_local_sources,file_name), os.R_OK) and not from_scratch:

IOError: Saved data at /home/king/.sage/pGroupCohomology/db/16gp2/H16gp2.sobj are not readable: NEWOBJ class argument isn't a type object

So, the previous error arising in lookup_global has gone.

I am afraid the new error looks more like an error that needs to be fixed in the package, not in the Sage library. We will see...

simon-king-jena commented 5 years ago
comment:126

Next step of debugging.

In Py2, where we still have py3-pickles in DOT_SAGE:

sage: from pGroupCohomology.cohomology import COHO
sage: H = COHO()
sage: S = load('/home/king/.sage/pGroupCohomology/db/16gp2/dat/State.sobj')
sage: H.__setstate__(S)
---------------------------------------------------------------------------
IOError                                   Traceback (most recent call last)
<ipython-input-4-9c79e2f6e0a6> in <module>()
----> 1 H.__setstate__(S)

pGroupCohomology/cohomology.pyx in pGroupCohomology.cohomology.COHO.__setstate__()

IOError: Unable to read resolution saved at /home/king/Sage/git/sage/local/share/pGroupCohomology/16gp2/dat/R16gp2.sobj

The same in py3:

sage: from pGroupCohomology.cohomology import COHO
sage: H = COHO()
sage: S = load('/home/king/.sage/pGroupCohomology/db/16gp2/dat/State.sobj')
sage: H.__setstate__(S)
sage: print(H)

Cohomology ring of Small Group number 2 of order 16 with coefficients in GF(2)

Computation complete
Minimal list of generators:
[c_2_1: 2-Cocycle in H^*(SmallGroup(16,2); GF(2)),
 c_2_2: 2-Cocycle in H^*(SmallGroup(16,2); GF(2)),
 c_1_0: 1-Cocycle in H^*(SmallGroup(16,2); GF(2)),
 c_1_1: 1-Cocycle in H^*(SmallGroup(16,2); GF(2))]
Minimal list of algebraic relations:
[c_1_0^2,
 c_1_1^2]

So, the error is now moved to __setstate__. I guess I'll fix it there.

simon-king-jena commented 5 years ago
comment:127

Meanwhile I know at what point the error mentioned in comment:125 arises:

sage: pg_RESL_sparse_unpickle_class = unpickle_global('pGroupCohomology.resolution', 'RESL_sparse_unpickle_class')
sage: si1 = unpickle_newobj(pg_RESL_sparse_unpickle_class, ())
---------------------------------------------------------------------------
UnpicklingError                           Traceback (most recent call last)
<ipython-input-2-855d0a850e70> in <module>()
----> 1 si1 = unpickle_newobj(pg_RESL_sparse_unpickle_class, ())

/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/misc/explain_pickle.pyc in unpickle_newobj(klass, args)
   2438         return pers[int(id)]
   2439 
-> 2440     return SageUnpickler.loads(pickle, persistent_load=pers_load)
   2441 
   2442 

/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.SageUnpickler.loads (build/cythonized/sage/misc/persist.c:6467)()
    962         """
    963 
--> 964         return cls(io.BytesIO(data), **kwargs).load()
    965 
    966 

/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/misc/persist.pyx in sage.misc.persist._BaseUnpickler.load (build/cythonized/sage/misc/persist.c:5806)()
    655 
    656         def load(self):
--> 657             return self._unpickler.load()
    658 
    659         def noload(self):

UnpicklingError: NEWOBJ class argument isn't a type object

But I do not have the faintest idea yet what this error means. For comparison, in py3 I get

sage: pg_RESL_sparse_unpickle_class = unpickle_global('pGroupCohomology.resolution', 'RESL_sparse_unpickle_class')
sage: si1 = unpickle_newobj(pg_RESL_sparse_unpickle_class, ())
sage: si1
<pGroupCohomology.resolution.RESL_sparse_unpickle_class object at 0x7f2ee987abe0>

I'm guessing that a class in Python-3 is a type, but in Python-2 it generally isn't. Is that the meaning of the error? But isn't it the case that a class defined in Cython (here: resolution.pyx) automatically is a type?

Anyway, I'll try to turn class RESL_sparse_unpickle_class into class RESL_sparse_unpickle_class(object) and see if it helps.

simon-king-jena commented 5 years ago
comment:128

Replying to @simon-king-jena:

Anyway, I'll try to turn class RESL_sparse_unpickle_class into class RESL_sparse_unpickle_class(object) and see if it helps.

Nope.

sage: from pGroupCohomology.resolution import RESL_sparse_unpickle_class
sage: RESL_sparse_unpickle_class
<class 'pGroupCohomology.resolution.RESL_sparse_unpickle_class'>
sage: isinstance(_, type)
True

but still

sage: pg_RESL_sparse_unpickle_class = unpickle_global('pGroupCohomology.resolution', 'RESL_sparse_unpickle_class')
sage: si1 = unpickle_newobj(pg_RESL_sparse_unpickle_class, ())
Traceback (most recent call last):
...
UnpicklingError: NEWOBJ class argument isn't a type object
simon-king-jena commented 5 years ago
comment:129

Odd. After trying again, the error is gone. In Py2:

sage: R = load("/home/king/.sage/pGroupCohomology/db/16gp2/dat/R16gp2.sobj")
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-1-c703eea54ef8> in <module>()
----> 1 R = load("/home/king/.sage/pGroupCohomology/db/16gp2/dat/R16gp2.sobj")

/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.load (build/cythonized/sage/misc/persist.c:2845)()
    157 
    158     ## Load file by absolute filename
--> 159     with open(filename, 'rb') as fobj:
    160         X = loads(fobj.read(), compress=compress, **kwargs)
    161     try:

/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.load (build/cythonized/sage/misc/persist.c:2795)()
    158     ## Load file by absolute filename
    159     with open(filename, 'rb') as fobj:
--> 160         X = loads(fobj.read(), compress=compress, **kwargs)
    161     try:
    162         X._default_filename = os.path.abspath(filename)

/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/misc/persist.pyx in sage.misc.persist.loads (build/cythonized/sage/misc/persist.c:6929)()
   1043 
   1044     unpickler = SageUnpickler(io.BytesIO(s), **kwargs)
-> 1045     return unpickler.load()
   1046 
   1047 

/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/misc/persist.pyx in sage.misc.persist._BaseUnpickler.load (build/cythonized/sage/misc/persist.c:5806)()
    655 
    656         def load(self):
--> 657             return self._unpickler.load()
    658 
    659         def noload(self):

pGroupCohomology/resolution.pyx in pGroupCohomology.resolution.RESL_sparse_unpickle_class.__call__()

pGroupCohomology/resolution.pyx in pGroupCohomology.resolution.RESL.__init__()

OSError: Bad file format in file pgroup.c (line 404)

So, now complain about NEWOBJ any more.

simon-king-jena commented 5 years ago
comment:130

Spoke too soon. Immediately after this different error (i.e., in the same session), I again got the "NEWOBJ" error.

simon-king-jena commented 5 years ago
comment:131
sage: from pGroupCohomology.resolution import RESL_sparse_unpickle_class
sage: pg_RESL_sparse_unpickle_class is RESL_sparse_unpickle_class
False

In a different session, I got the answer "True".

Anyway, the error about "NEWOBJ" comes from python. It seems that python decided to kill data persistency. I tried to work around to the extent that py2-pickles of cohomology rings seem to be legible in py3. But I think I will not work further on making py3-pickles legible in py2.

Another question: Why is the branch of this ticket shown red?

simon-king-jena commented 5 years ago
comment:132

The oddity is: Sometimes pg_RESL_sparce_unpickle_class is RESL_sparse_unpickle_class and sometimes it isn't. When the error occurs, I get

sage: pg_RESL_sparse_unpickle_class
<class pGroupCohomology.auxiliaries.unpickle_old_resl at 0x7fb8d4f549a8>

It is not nice that unpickle_global('pGroupCohomology.resolution', 'RESL_sparse_unpickle_class') sometimes returns something else. But perhaps it is worth trying to make this class inherit from object.

simon-king-jena commented 5 years ago
comment:133

Replying to @simon-king-jena:

But perhaps it is worth trying to make this class inherit from object.

Nope, didn't help. As there doesn't seem to be an easy fix, I do not plan to work further on it.

Do I understand correctly that I have to rebase on the current develop branch?

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 5 years ago

Changed commit from 463806c to 7ad6a4a

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 5 years ago

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

7ad6a4aUpgrade p_group_cohomology to version 3.3
jhpalmieri commented 5 years ago
comment:135

Maybe it would be simpler, if any error arises reading a pickle, to just delete and reproduce the pickle file. Compatibility going from Python 3 to Python 2 may be annoying to debug and to fix, and besides, there may be other errors that we haven't foreseen.

jhpalmieri commented 5 years ago
comment:136

Or as you say, to just not work on the Py3 -> Py2 conversion. That's fine with me.

simon-king-jena commented 5 years ago
comment:137

I am using unpickle_override for RESL_sparse_unpickle_class, but I am not sure if that's needed at all.

simon-king-jena commented 5 years ago
comment:138

Bummer. Apparently I have destroyed the git repository for my py3 installation. It says:

king@klap:~/Sage/git/py3$ git lg
fatal: your current branch 't/28444/fix_backwards_incompatibility_of_unpickling_in_python_3' does not have any commits yet

and ALL files in the Sage tree are untracked. How has that happened, and how to fix it?

dimpase commented 5 years ago
comment:139

Hopefully John has a version of this branch just before you did that fateful forced push.

There could be copies left on patchbots, maybe LinuxMint/18.2/x86_64/4.8.0-53-generic/pc78-math patchbot does have it...

jhpalmieri commented 5 years ago
comment:140

Unfortunately I don't, but there are the commits in comment:113 which are still accessible.

dimpase commented 5 years ago
comment:141

yes, indeed, these commits are available in the full "git fetch trac" - I have them locally, in case they get gc'd.

simon-king-jena commented 5 years ago
comment:142

Replying to @dimpase:

Hopefully John has a version of this branch just before you did that fateful forced push.

  1. What's wrong with the forced push? IMHO, that has nothing to to with the problem whatsoever. The forced push is nothing more than a rebasing of the previously committed branch. After pushing it, I could still work.
  2. The branch is still present in my git repository. Perhaps I didn't make it clear. I have a single git repository with two worktrees, one for py2, one for py3. The py2 worktree is fine (AFAIC), in particular I can checkout any branch. However, it seems that I deleted the branch on which the py3 worktree was living, with git branch -d. As a result, all files in the py3 worktree are now untracked, as is shown by git status, and git log says that there has never been a commit. Again, I am talking about only one of my two worktrees.
  3. The deleted branch is the branch from #28444, not from here. That branch is merged in "develop", that's why I did git branch -d on it.

So, from my perspective, "solving the problem" means "allowing me to do git checkout <branchname> in the py3-worktree" -- this currently is refused, because git claims that I should first commit or stash the whole Sage library.

simon-king-jena commented 5 years ago
comment:143

Would git checkout -f <branch_name> possibly help? Or would I possibly destroy even more?