search

sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.33k stars 453 forks source link

Metaticket: Add support for python 3.6+ #15530

Closed ohanar closed 3 years ago

ohanar commented 10 years ago

How to build sage with python3:

make configure
./configure --with-python=3
make

Beware that the second step above should be redone after every make distclean.

For progress on making all doctests pass, see #26212 and #28298

In order to support python 3.7, the following needs to be fixed:

  1. 15510 - upgrade to a newer version of setuptools

  2. 15511 - upgrade to a newer version of rpy2

  3. 15512 - upgrade to a newer version of sympy

  4. 14854 - upgrade to a newer version of pycrypto

  5. 15532 - upgrade to a newer version of networkx

  6. 15537 - fix csage to work with python3

  7. 15539 - switch from using PIL to Pillow

  8. 15540 - trivial python3 fixes to a few spkgs

  9. 15541 - fix sage-location and sage-download-file for python3

  10. 15593 - remove sqlalchemy

  11. 17591 - remove gdmodule

  12. 15620 - Stop using StandardError

  13. 15755 - upgrade cython to version 0.20.1

  14. 15807 - upgrade mpmath to version 0.18

  15. 15980 - meta-ticket for python3 compatibility of the sage library (stage 1)

  16. 16052 - meta-ticket for python3 compatibility of the sage library (stage 2)

  17. 17854 - remove c_lib

  18. 18437 - fix polybori and python 3

  19. 18537 - fix Pynac at least building against python 3

  20. 17607 - add python 3 package

  21. 27024 - make sagetex python3 compatible

Possibly relevant timeline from PEP 373:

Being the last of the 2.x series, 2.7 will have an extended period of maintenance. The current plan is to support it for at least 10 years from the initial 2.7 release. This means there will be bugfix releases until 2020.

Depends on #15510 Depends on #15511 Depends on #15512 Depends on #15531 Depends on #15532 Depends on #15537 Depends on #15539 Depends on #15540 Depends on #15541 Depends on #15807 Depends on #15980 Depends on #18537 Depends on #16052 Depends on #18437 Depends on #27024

Component: python3

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/15530

ohanar commented 10 years ago

Description changed:

--- 
+++ 
@@ -3,3 +3,4 @@
 1. #15510 - upgrade to a newer version of setuptools
 2. #15511 - upgrade to a newer version of rpy2
 3. #15512 - upgrade to a newer version of sympy
+4. #15531 - upgrade to a newer version of pycrypto
ohanar commented 10 years ago

Changed dependencies from #15510, #15511, #15512 to #15510, #15511, #15512, #15531

ohanar commented 10 years ago

Description changed:

--- 
+++ 
@@ -4,3 +4,4 @@
 2. #15511 - upgrade to a newer version of rpy2
 3. #15512 - upgrade to a newer version of sympy
 4. #15531 - upgrade to a newer version of pycrypto
+5. #15532 - upgrade to a newer version of networkx
ohanar commented 10 years ago

Changed dependencies from #15510, #15511, #15512, #15531 to #15510, #15511, #15512, #15531, #15532

ohanar commented 10 years ago

Description changed:

--- 
+++ 
@@ -5,3 +5,4 @@
 3. #15512 - upgrade to a newer version of sympy
 4. #15531 - upgrade to a newer version of pycrypto
 5. #15532 - upgrade to a newer version of networkx
+6. #15537 - fix csage to work with python3
ohanar commented 10 years ago

Changed dependencies from #15510, #15511, #15512, #15531, #15532 to #15510, #15511, #15512, #15531, #15532, #15537

ohanar commented 10 years ago

Description changed:

--- 
+++ 
@@ -6,3 +6,4 @@
 4. #15531 - upgrade to a newer version of pycrypto
 5. #15532 - upgrade to a newer version of networkx
 6. #15537 - fix csage to work with python3
+7. #15539 - switch from using PIL to Pillow
ohanar commented 10 years ago

Changed dependencies from #15510, #15511, #15512, #15531, #15532, #15537 to #15510, #15511, #15512, #15531, #15532, #15537, #15539

ohanar commented 10 years ago

Description changed:

--- 
+++ 
@@ -7,3 +7,4 @@
 5. #15532 - upgrade to a newer version of networkx
 6. #15537 - fix csage to work with python3
 7. #15539 - switch from using PIL to Pillow
+8. #15540 - trivial python3 fixes to a few spkgs
ohanar commented 10 years ago

Changed dependencies from #15510, #15511, #15512, #15531, #15532, #15537, #15539 to #15510, #15511, #15512, #15531, #15532, #15537, #15539, #15540

ohanar commented 10 years ago

Description changed:

--- 
+++ 
@@ -8,3 +8,4 @@
 6. #15537 - fix csage to work with python3
 7. #15539 - switch from using PIL to Pillow
 8. #15540 - trivial python3 fixes to a few spkgs
+9. #15541 - fix sage-location and sage-download-file for python3
ohanar commented 10 years ago

Changed dependencies from #15510, #15511, #15512, #15531, #15532, #15537, #15539, #15540 to #15510, #15511, #15512, #15531, #15532, #15537, #15539, #15540, #15541

tscrim commented 10 years ago
comment:8

It looks like everything is done. Now we just need to switch Sage to python 3...

ohanar commented 10 years ago
comment:9

Sorry, I should have put a note that I haven't listed all issues (since I haven't even encountered all the issues yet).

ohanar commented 10 years ago

Description changed:

--- 
+++ 
@@ -9,3 +9,6 @@
 7. #15539 - switch from using PIL to Pillow
 8. #15540 - trivial python3 fixes to a few spkgs
 9. #15541 - fix sage-location and sage-download-file for python3
+10. upgrade scons to a version that supports python3 (does not yet exist)
+11. upgrade polybory to a version that supports python3 (does not yet exist)
+12. other issues as encountered
ohanar commented 10 years ago

Description changed:

--- 
+++ 
@@ -9,6 +9,7 @@
 7. #15539 - switch from using PIL to Pillow
 8. #15540 - trivial python3 fixes to a few spkgs
 9. #15541 - fix sage-location and sage-download-file for python3
-10. upgrade scons to a version that supports python3 (does not yet exist)
-11. upgrade polybory to a version that supports python3 (does not yet exist)
-12. other issues as encountered
+10. #15593 - sqlalchemy (the version we ship) and gdmodule are incompatible with python3
+11. upgrade scons to a version that supports python3 (does not yet exist)
+12. upgrade polybory to a version that supports python3 (does not yet exist)
+13. other issues as encountered
ohanar commented 10 years ago

Description changed:

--- 
+++ 
@@ -3,7 +3,7 @@
 1. #15510 - upgrade to a newer version of setuptools
 2. #15511 - upgrade to a newer version of rpy2
 3. #15512 - upgrade to a newer version of sympy
-4. #15531 - upgrade to a newer version of pycrypto
+4. #14854 - upgrade to a newer version of pycrypto
 5. #15532 - upgrade to a newer version of networkx
 6. #15537 - fix csage to work with python3
 7. #15539 - switch from using PIL to Pillow
ohanar commented 10 years ago

Description changed:

--- 
+++ 
@@ -13,3 +13,8 @@
 11. upgrade scons to a version that supports python3 (does not yet exist)
 12. upgrade polybory to a version that supports python3 (does not yet exist)
 13. other issues as encountered
+
+Not essential, but useful tickets:
+
+1. #15594 - use autotools for csage
+2. #15604 - remove csage's psuedo-dependencies pynac and polybori
ohanar commented 10 years ago

Description changed:

--- 
+++ 
@@ -10,9 +10,10 @@
 8. #15540 - trivial python3 fixes to a few spkgs
 9. #15541 - fix sage-location and sage-download-file for python3
 10. #15593 - sqlalchemy (the version we ship) and gdmodule are incompatible with python3
-11. upgrade scons to a version that supports python3 (does not yet exist)
-12. upgrade polybory to a version that supports python3 (does not yet exist)
-13. other issues as encountered
+11. #15620 - Stop using `StandardError`
+12. upgrade scons to a version that supports python3 (does not yet exist)
+13. upgrade polybory to a version that supports python3 (does not yet exist)
+14. other issues as encountered

 Not essential, but useful tickets:
jdemeyer commented 10 years ago

Description changed:

--- 
+++ 
@@ -11,8 +11,8 @@
 9. #15541 - fix sage-location and sage-download-file for python3
 10. #15593 - sqlalchemy (the version we ship) and gdmodule are incompatible with python3
 11. #15620 - Stop using `StandardError`
-12. upgrade scons to a version that supports python3 (does not yet exist)
-13. upgrade polybory to a version that supports python3 (does not yet exist)
+12. upgrade scons to a version that supports python3 (does not yet exist) or get rid of scons
+13. upgrade polybori to a version that supports python3 (does not yet exist)
 14. other issues as encountered

 Not essential, but useful tickets:
ohanar commented 10 years ago

Description changed:

--- 
+++ 
@@ -11,9 +11,10 @@
 9. #15541 - fix sage-location and sage-download-file for python3
 10. #15593 - sqlalchemy (the version we ship) and gdmodule are incompatible with python3
 11. #15620 - Stop using `StandardError`
-12. upgrade scons to a version that supports python3 (does not yet exist) or get rid of scons
-13. upgrade polybori to a version that supports python3 (does not yet exist)
-14. other issues as encountered
+12. #15755 - upgrade cython to version 0.20.1
+13. upgrade scons to a version that supports python3 (does not yet exist) or get rid of scons
+14. upgrade polybori to a version that supports python3 (does not yet exist)
+15. other issues as encountered

 Not essential, but useful tickets:
ohanar commented 10 years ago

Description changed:

--- 
+++ 
@@ -12,9 +12,10 @@
 10. #15593 - sqlalchemy (the version we ship) and gdmodule are incompatible with python3
 11. #15620 - Stop using `StandardError`
 12. #15755 - upgrade cython to version 0.20.1
-13. upgrade scons to a version that supports python3 (does not yet exist) or get rid of scons
-14. upgrade polybori to a version that supports python3 (does not yet exist)
-15. other issues as encountered
+13. #15807 - upgrade mpmath to version 0.18
+14. upgrade scons to a version that supports python3 (does not yet exist) or get rid of scons
+15. upgrade polybori to a version that supports python3 (does not yet exist)
+16. other issues as encountered

 Not essential, but useful tickets:
ohanar commented 10 years ago

Changed dependencies from #15510, #15511, #15512, #15531, #15532, #15537, #15539, #15540, #15541 to #15510, #15511, #15512, #15531, #15532, #15537, #15539, #15540, #15541, #15807

ohanar commented 10 years ago

Description changed:

--- 
+++ 
@@ -13,9 +13,10 @@
 11. #15620 - Stop using `StandardError`
 12. #15755 - upgrade cython to version 0.20.1
 13. #15807 - upgrade mpmath to version 0.18
-14. upgrade scons to a version that supports python3 (does not yet exist) or get rid of scons
-15. upgrade polybori to a version that supports python3 (does not yet exist)
-16. other issues as encountered
+14. #15980 - meta-ticket for python3 compatibility of the sage library
+15. upgrade scons to a version that supports python3 (does not yet exist) or get rid of scons
+16. upgrade polybori to a version that supports python3 (does not yet exist)
+17. other issues as encountered

 Not essential, but useful tickets:
ohanar commented 10 years ago

Changed dependencies from #15510, #15511, #15512, #15531, #15532, #15537, #15539, #15540, #15541, #15807 to #15510, #15511, #15512, #15531, #15532, #15537, #15539, #15540, #15541, #15807, #15980

3a5ab40a-5e15-47f8-8025-1665d51c0660 commented 10 years ago

Description changed:

--- 
+++ 
@@ -13,10 +13,11 @@
 11. #15620 - Stop using `StandardError`
 12. #15755 - upgrade cython to version 0.20.1
 13. #15807 - upgrade mpmath to version 0.18
-14. #15980 - meta-ticket for python3 compatibility of the sage library
-15. upgrade scons to a version that supports python3 (does not yet exist) or get rid of scons
-16. upgrade polybori to a version that supports python3 (does not yet exist)
-17. other issues as encountered
+14. #15980 - meta-ticket for python3 compatibility of the sage library (stage 1)
+15. #16052 - meta-ticket for python3 compatibility of the sage library (stage 2)
+16. upgrade scons to a version that supports python3 (does not yet exist) or get rid of scons
+17. upgrade polybori to a version that supports python3 (does not yet exist)
+18. other issues as encountered

 Not essential, but useful tickets:
jasongrout commented 10 years ago
comment:20

Python 3.4 is out now...

(really just commenting to subscribe to updates)

kcrisman commented 9 years ago

Description changed:

--- 
+++ 
@@ -23,3 +23,11 @@

 1. #15594 - use autotools for csage
 2. #15604 - remove csage's psuedo-dependencies pynac and polybori
+
+Possibly relevant timeline from [here](https://www.python.org/dev/peps/pep-0373/):
+> Being the last of the 2.x series, 2.7 will have an extended period of maintenance. The current plan is to support it for at least 10 years from the initial 2.7 release. This means there will be bugfix releases until 2020.
+> Planned future release dates:
+> * 2.7.9rc1 2014-11-26
+> * 2.7.9 2014-12-10
+> * 2.7.10 June 2015
+> beyond this date, releases as needed
jdemeyer commented 9 years ago

Description changed:

--- 
+++ 
@@ -21,8 +21,7 @@

 Not essential, but useful tickets:

-1. #15594 - use autotools for csage
-2. #15604 - remove csage's psuedo-dependencies pynac and polybori
+1. #17484 - use distutils to compile libcsage 

 Possibly relevant timeline from [here](https://www.python.org/dev/peps/pep-0373/):
 > Being the last of the 2.x series, 2.7 will have an extended period of maintenance. The current plan is to support it for at least 10 years from the initial 2.7 release. This means there will be bugfix releases until 2020.
jdemeyer commented 9 years ago

Description changed:

--- 
+++ 
@@ -15,9 +15,8 @@
 13. #15807 - upgrade mpmath to version 0.18
 14. #15980 - meta-ticket for python3 compatibility of the sage library (stage 1)
 15. #16052 - meta-ticket for python3 compatibility of the sage library (stage 2)
-16. upgrade scons to a version that supports python3 (does not yet exist) or get rid of scons
+16. upgrade scons to a version that supports python3 (does not yet exist) or get rid of scons (assuming #17484, scons is only used by polybori)
 17. upgrade polybori to a version that supports python3 (does not yet exist)
-18. other issues as encountered

 Not essential, but useful tickets:
jdemeyer commented 9 years ago

Description changed:

--- 
+++ 
@@ -22,10 +22,14 @@

 1. #17484 - use distutils to compile libcsage 

+---
+
 Possibly relevant timeline from [here](https://www.python.org/dev/peps/pep-0373/):
-> Being the last of the 2.x series, 2.7 will have an extended period of maintenance. The current plan is to support it for at least 10 years from the initial 2.7 release. This means there will be bugfix releases until 2020.
-> Planned future release dates:
-> * 2.7.9rc1 2014-11-26
-> * 2.7.9 2014-12-10
-> * 2.7.10 June 2015
-> beyond this date, releases as needed
+
+Being the last of the 2.x series, 2.7 will have an extended period of maintenance. The current plan is to support it for at least 10 years from the initial 2.7 release. This means there will be bugfix releases until 2020.
+
+Planned future release dates:
+
+* 2.7.10 June 2015
+* beyond this date, releases as needed
+
jdemeyer commented 9 years ago

Description changed:

--- 
+++ 
@@ -24,7 +24,7 @@

 ---

-Possibly relevant timeline from [here](https://www.python.org/dev/peps/pep-0373/):
+Possibly relevant timeline from [PEP 373](https://www.python.org/dev/peps/pep-0373/):

 Being the last of the 2.x series, 2.7 will have an extended period of maintenance. The current plan is to support it for at least 10 years from the initial 2.7 release. This means there will be bugfix releases until 2020.
ohanar commented 9 years ago

Description changed:

--- 
+++ 
@@ -9,14 +9,15 @@
 7. #15539 - switch from using PIL to Pillow
 8. #15540 - trivial python3 fixes to a few spkgs
 9. #15541 - fix sage-location and sage-download-file for python3
-10. #15593 - sqlalchemy (the version we ship) and gdmodule are incompatible with python3
-11. #15620 - Stop using `StandardError`
-12. #15755 - upgrade cython to version 0.20.1
-13. #15807 - upgrade mpmath to version 0.18
-14. #15980 - meta-ticket for python3 compatibility of the sage library (stage 1)
-15. #16052 - meta-ticket for python3 compatibility of the sage library (stage 2)
-16. upgrade scons to a version that supports python3 (does not yet exist) or get rid of scons (assuming #17484, scons is only used by polybori)
-17. upgrade polybori to a version that supports python3 (does not yet exist)
+10. #15593 - remove sqlalchemy
+11. #17591 - remove gdmodule
+12. #15620 - Stop using `StandardError`
+13. #15755 - upgrade cython to version 0.20.1
+14. #15807 - upgrade mpmath to version 0.18
+15. #15980 - meta-ticket for python3 compatibility of the sage library (stage 1)
+16. #16052 - meta-ticket for python3 compatibility of the sage library (stage 2)
+17. upgrade scons to a version that supports python3 (does not yet exist) or get rid of scons (assuming #17484, scons is only used by polybori)
+18. upgrade polybori to a version that supports python3 (does not yet exist)

 Not essential, but useful tickets:
ohanar commented 9 years ago

Description changed:

--- 
+++ 
@@ -18,6 +18,7 @@
 16. #16052 - meta-ticket for python3 compatibility of the sage library (stage 2)
 17. upgrade scons to a version that supports python3 (does not yet exist) or get rid of scons (assuming #17484, scons is only used by polybori)
 18. upgrade polybori to a version that supports python3 (does not yet exist)
+19. #17607 - add python 3 package

 Not essential, but useful tickets:
kiwifb commented 9 years ago
comment:31

We had to abandon the idea of using distutils for libcsage at least as is. We could certainly use getting rid of scons for it. Do we know if there is anyone currently active for polybori? My understanding is that Alexander Dreyer has gone to the industry and no one has really taken over polybori.

ohanar commented 9 years ago
comment:32

Replying to @kiwifb:

We had to abandon the idea of using distutils for libcsage at least as is.

There was also a autotools approach at #15594, which I think could be returned to (it was closed as won't fix in favor of distutils).

We could certainly use getting rid of scons for it. Do we know if there is anyone currently active for polybori? My understanding is that Alexander Dreyer has gone to the industry and no one has really taken over polybori.

I don't really know the situation, from an outsiders perspective, it certainly looks as if the project is dead. I don't really know how best to deal with it.

kiwifb commented 9 years ago
comment:33

Yes we closed the autotools ticket when we decided to go distutils. And then we found that distutils couldn't be used that way. So we should try to revive that.

Someone should try to contact Alexander or Michael Brickenstein to see if they will make any new releases. As I see it the first problem will be to remove scons as the build system for polybori. Autotools+distutils feels like the right option for it. Then we could work out the python3 migration. Removing polybori would be interesting too but I think it is ingrained deep in sage.

ohanar commented 9 years ago
comment:34

Replying to @kiwifb:

Someone should try to contact Alexander or Michael Brickenstein to see if they will make any new releases. As I see it the first problem will be to remove scons as the build system for polybori. Autotools+distutils feels like the right option for it. Then we could work out the python3 migration. Removing polybori would be interesting too but I think it is ingrained deep in sage.

I don't think it is too deeply integrated into sage, it seems like it is mainly used for boolean polynomial rings (which themselves don't seem to be very widely used in the sage library). If it is easier to make a version of sage compatible with python 3 which doesn't have polybori, than it is to bring polybori up to speed, I would definitely vote for the former (especially if it is significantly easier, which I suspect might be the case ... but I could be completely wrong). In either case, I would keep polybori around for the python 2 version of sage.

jdemeyer commented 9 years ago

Description changed:

--- 
+++ 
@@ -16,7 +16,7 @@
 14. #15807 - upgrade mpmath to version 0.18
 15. #15980 - meta-ticket for python3 compatibility of the sage library (stage 1)
 16. #16052 - meta-ticket for python3 compatibility of the sage library (stage 2)
-17. upgrade scons to a version that supports python3 (does not yet exist) or get rid of scons (assuming #17484, scons is only used by polybori)
+17. upgrade scons to a version that supports python3 (does not yet exist) or get rid of scons (assuming #17854, scons is only used by polybori)
 18. upgrade polybori to a version that supports python3 (does not yet exist)
 19. #17607 - add python 3 package
jdemeyer commented 9 years ago

Description changed:

--- 
+++ 
@@ -20,10 +20,6 @@
 18. upgrade polybori to a version that supports python3 (does not yet exist)
 19. #17607 - add python 3 package

-Not essential, but useful tickets:
-
-1. #17484 - use distutils to compile libcsage 
-
 ---

 Possibly relevant timeline from [PEP 373](https://www.python.org/dev/peps/pep-0373/):
kiwifb commented 9 years ago
comment:37

Something for which there is no ticket yet but shouldn't be forgotten. pynac is python2.7 only last time I checked. ginac/numeric.cpp needs a significant overhaul to move to python3, some of which I think I could do but some other like the rich cmp I wouldn't.

kcrisman commented 9 years ago
comment:38

Something for which there is no ticket yet but shouldn't be forgotten. pynac is python2.7 only last time I checked. ginac/numeric.cpp needs a significant overhaul to move to python3, some of which I think I could do but some other like the rich cmp I wouldn't.

rws, you've been doing great work recently with a lot of the nitty-gritty stuff in Pynac - any ideas? We could at least open a ticket if this is not too impossible.

ohanar commented 9 years ago
comment:39

I was looking at porting pynac to python 3 a few weeks ago, and it doesn't look too bad (even the rich comp stuff, but I could always be wrong). I was either going to work on this or see if I can't figure something to do with polybori and python 3 during Sage Days 64.25.

rwst commented 9 years ago
comment:40

I'm only starting now to get into Pynac's Python interface. I also have no idea which changes are necessary because of Python 3. So, if you cannot give me at least a lead by opening a detailed Pynac ticket then don't expect rapid progress there.

kiwifb commented 9 years ago
comment:41

OK so I opened a pynac issue at https://github.com/pynac/pynac/issues/42 and I'll start making pull requests for what I can fix as I go along.

rwst commented 9 years ago
comment:42

The pynac issue is now fixed in pynac master, pending a full Py3 doctest of Sage.

rwst commented 9 years ago

Changed dependencies from #15510, #15511, #15512, #15531, #15532, #15537, #15539, #15540, #15541, #15807, #15980 to #15510, #15511, #15512, #15531, #15532, #15537, #15539, #15540, #15541, #15807, #15980, pynac-0.3.8

kiwifb commented 9 years ago
comment:43

Pending insulation of polybori or making it python3.4 compatible and modest hacking (in pynac and libcsage) I should be able to make a version of sage-on-gentoo that will install sage for both python 2.7 and 3.4 and test things further. I see polybori as the biggest hurdle, finishing the integration of libcsage would help too but polybori is really the biggest problem as far as I can see.

vbraun commented 9 years ago
comment:44

There is some progress on scons to move to python 3 (http://thread.gmane.org/gmane.comp.programming.tools.scons.devel/12929). Still, I think thats the least of the problems. If scons doesn't get support in time we can just build Python 2 + Python 3 and use python2 for scons at build time.

kiwifb commented 9 years ago
comment:45

Replying to @vbraun:

There is some progress on scons to move to python 3 (http://thread.gmane.org/gmane.comp.programming.tools.scons.devel/12929). Still, I think thats the least of the problems. If scons doesn't get support in time we can just build Python 2 + Python 3 and use python2 for scons at build time.

I agree in principle but scons is potentially not the only problem in polybori.