update python3 spkg to version 3.6.1

[fchapoton]

needed for #22305

https://www.python.org/ftp/python/3.6.1/Python-3.6.1.tgz

renamed to Python-3.6.1.tar.gz

Depends on #22582

CC: @jdemeyer @embray @tscrim

Component: python3

Author: John Palmieri

Branch/Commit: 7c393d7

Reviewer: Frédéric Chapoton

Issue created by migration from https://trac.sagemath.org/ticket/22942

[jhpalmieri]

comment:1

Adding #22582 as a dependency because it makes some changes to the python3 spkg-install file.

[jhpalmieri]

Dependencies: #22582

[fchapoton]

comment:2

and this is probably going to interfere with Cygwin, sigh..

[fchapoton]

Description changed:

--- 
+++ 
@@ -1 +1,3 @@
 needed for #22305
+
+https://www.python.org/ftp/python/3.6.1/Python-3.6.1.tar.xz

[fchapoton]

Description changed:

--- 
+++ 
@@ -1,3 +1,3 @@
 needed for #22305

-https://www.python.org/ftp/python/3.6.1/Python-3.6.1.tar.xz
+https://www.python.org/ftp/python/3.6.1/Python-3.6.1.tgz

[fchapoton]

Description changed:

--- 
+++ 
@@ -1,3 +1,5 @@
 needed for #22305

 https://www.python.org/ftp/python/3.6.1/Python-3.6.1.tgz
+
+renamed to Python-3.6.1.tar.gz

[fchapoton]

comment:6

maybe our patches will need to be updated

---

New commits:

644e944

python3 update to 3.6.1

[fchapoton]

Commit: 644e944

[fchapoton]

Branch: public/python3.6.1

[fchapoton]

comment:7

indeed

[python3-3.6.1] Error applying '../patches/3.5.2-struct_siginfo_si_band.patch'

[fchapoton]

comment:8

An interesting bunch of patches here:

https://github.com/cygwinports/python3

[embray]

comment:9

Most of the existing patches being used come from there. A few come directly from bugs.python.org issues.

[fchapoton]

comment:10

I am not able to do the required job: refresh the patches, maybe remove some, or add others.

So I would appreciate if someone else can take care of this ticket.

[jhpalmieri]

comment:11

With #22756, #22582, and the following changes, this builds for me on OS X. I'm not sure about the change to 3.5.2-struct_siginfo_si_band.patch, but the other change makes sense: our patch (which we took from a Python bug report) is now included in version 3.6.1.

diff --git a/build/pkgs/python3/patches/3.5.2-struct_siginfo_si_band.patch b/build/pkgs/python3/patches/3.5.2-struct_siginfo_si_band.patch
index c2ee610..2848c9e 100644
--- a/build/pkgs/python3/patches/3.5.2-struct_siginfo_si_band.patch
+++ b/build/pkgs/python3/patches/3.5.2-struct_siginfo_si_band.patch
@@ -39,15 +39,12 @@ index 2fbe259..8281f95 100755
  { $as_echo "$as_me:${as_lineno-$LINENO}: checking for time.h that defines altzone" >&5
  $as_echo_n "checking for time.h that defines altzone... " >&6; }
 diff --git a/configure.ac b/configure.ac
-index 8ef1760..6bfee80 100644
---- a/configure.ac
-+++ b/configure.ac
-@@ -3692,7 +3692,9 @@
- AC_CHECK_MEMBERS([struct stat.st_flags])
- AC_CHECK_MEMBERS([struct stat.st_gen])
- AC_CHECK_MEMBERS([struct stat.st_birthtime])
--AC_STRUCT_ST_BLOCKS
-+AC_CHECK_MEMBERS([struct stat.st_blocks])
+--- a/configure.ac     2017-05-04 10:49:21.000000000 -0700
++++ b/configure.ac     2017-05-04 10:49:50.000000000 -0700
+@@ -3930,6 +3930,8 @@
+   #include <sys/types.h>
+   #include <pwd.h>
+ ]])
 +# Issue #21085: In Cygwin, siginfo_t does not have si_band field.
 +AC_CHECK_MEMBERS([siginfo_t.si_band], [], [], [[#include <signal.h>]])

diff --git a/build/pkgs/python3/patches/pyport-apple-c++.patch b/build/pkgs/python3/patches/pyport-apple-c++.patch
deleted file mode 100644
index 61aea6e..0000000
--- a/build/pkgs/python3/patches/pyport-apple-c++.patch
+++ /dev/null
@@ -1,23 +0,0 @@
---- a/Include/pyport.h 2017-04-07 13:26:39.000000000 -0700
-+++ b/Include/pyport.h 2017-04-07 13:27:54.000000000 -0700
-@@ -688,6 +688,12 @@
- #endif
- 
- #ifdef _PY_PORT_CTYPE_UTF8_ISSUE
-+#ifndef __cplusplus
-+   /* The workaround below is unsafe in C++ because
-+    * the <locale> defines these symbols as real functions,
-+    * with a slightly different signature.
-+    * See issue #10910
-+    */
- #include <ctype.h>
- #include <wctype.h>
- #undef isalnum
-@@ -705,6 +711,7 @@
- #undef toupper
- #define toupper(c) towupper(btowc(c))
- #endif
-+#endif
- 
- 
- /* Declarations for symbol visibility.

At some point it would be good to refresh the patches, since some apply with some fuzz.

[embray]

comment:12

Looks fine to me !^

[jhpalmieri]

comment:13

Okay, I will prepare a proper branch.

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 644e944 to 3821151

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

f54941a	Merge branch 'develop' into t/22942/public/python3.6.1
3821151	trac 22942: update python3 patches

[fchapoton]

comment:16

It fails on enum34 with SAGE_PYTHON3=yes, because

The basic issue is that enum34 is not meant to be installed in Python versions 3.4+. 

Damn!!!

REF: https://bitbucket.org/stoneleaf/enum34/issues/19/enum34-isnt-compatible-with-python-36

[jhpalmieri]

comment:17

It looks like enum34 is only used by traitlets. Does traitlets need it with Python 3.6, or does Python 3.6 provide the necessary functionality on its own?

[jhpalmieri]

comment:18

First, enum34 built for me with SAGE_PYTHON3=yes. Second, anyway, we could make this change to its spkg-install file:

diff --git a/build/pkgs/enum34/spkg-install b/build/pkgs/enum34/spkg-install
index c1a2289ade..0f465dfd7e 100755
--- a/build/pkgs/enum34/spkg-install
+++ b/build/pkgs/enum34/spkg-install
@@ -1,3 +1,7 @@
 #!/usr/bin/env bash

-cd src && $PIP_INSTALL .
+if [ "$SAGE_PYTHON3" = "yes" ]; then
+    echo "Not installing enum34: not necessary with Python 3.5 or later."
+else
+    cd src && $PIP_INSTALL .
+fi

I tried this and traitlets built without it: I think traitlets only really needs it with Python 3.4 or earlier. I'll try a complete build with this patch to see how things go.

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

2bc12b6

trac 22942: do not build enum34 with Python 3.5 or later

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 3821151 to 2bc12b6

[jhpalmieri]

comment:20

This worked for me, so I've added it to the branch.

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

0f0ed1c

trac 22942: do not build enum34 with Python 3.5 or later

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 2bc12b6 to 0f0ed1c

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 0f0ed1c to 45e44a0

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

45e44a0

trac 22942: do not build enum34 with Python 3.4 or later

[jhpalmieri]

comment:23

Alternatively, we could use https://github.com/jstasiak/enum-compat. But since we have complete control over our Python versions, I don't think we need to do that.

[fchapoton]

comment:24

The latest branch works for me with SAGE_PYTHON3=yes.

[fchapoton]

comment:25

and also works with python2

[fchapoton]

comment:26

I am therefore tempted to give a positive review.

@embray and others, do you have any objection, related to Cygwin or not ?

[fchapoton]

comment:27

I am setting to positive. If somebody objects, it's time to speak.

[fchapoton]

Author: John Palmieri

[fchapoton]

Reviewer: Frédéric Chapoton

[vbraun]

comment:28

The new python version improved syntax checking it seems:

$ sage -t --long src/sage/tests/py3_syntax.py
Running doctests with ID 2017-05-14-10-13-07-3c005bd2.
Git branch: develop
Using --optional=mpir,python2,sage
Doctesting 1 file.
sage -t --long --warn-long 72.0 src/sage/tests/py3_syntax.py
**********************************************************************
File "src/sage/tests/py3_syntax.py", line 17, in sage.tests.py3_syntax
Failed example:
    py3_syntax.run_tests('.py')   # long time
Expected nothing
Got:
    Invalid Python 3 syntax found:
      File "src/sage/calculus/calculus.py", line 2205
        global _syms
        ^
    SyntaxError: name '_syms' is used prior to global declaration
    <BLANKLINE>
    <BLANKLINE>
**********************************************************************
1 item had failures:
   1 of   4 in sage.tests.py3_syntax
    [29 tests, 1 failure, 18.18 s]
----------------------------------------------------------------------
sage -t --long --warn-long 72.0 src/sage/tests/py3_syntax.py  # 1 doctest failed
----------------------------------------------------------------------
Total time for all tests: 18.4 seconds
    cpu time: 1.4 seconds
    cumulative wall time: 18.2 seconds

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Changed commit from 45e44a0 to 7c393d7

[7ed8c4ca-6d56-4ae9-953a-41e42b4ed313]

Branch pushed to git repo; I updated commit sha1. New commits:

c82f856	Merge branch 'public/python3.6.1' in 8.0.b6
7c393d7	trac 22941 fixing syntax in calculus.py

[fchapoton]

comment:30

Indeed. Fixed.

Volker, can I set to positive in order to try again ?

[jhpalmieri]

comment:33

I seem to get doctest failures with this change. I think it would be better to move the definition of _syms earlier in the file. I'll try both ways more methodically, though.

diff --git a/src/sage/calculus/calculus.py b/src/sage/calculus/calculus.py
index 38a2ef7151..33204a5dee 100644
--- a/src/sage/calculus/calculus.py
+++ b/src/sage/calculus/calculus.py
@@ -1991,8 +1991,22 @@ _inverse_laplace = function_factory('ilt',

 ######################################i################

+# Parser for symbolic ring elements

+# We keep two dictionaries syms_cur and syms_default to keep the current symbol
+# table and the state of the table at startup respectively. These are used by
+# the restore() function (see sage.misc.reset).
+#
+# The dictionary _syms is used as a lookup table for the system function
+# registry by _find_func() below. It gets updated by
+# symbolic_expression_from_string() before calling the parser.
+_syms = syms_cur = symbol_table.get('functions', {})
+syms_default = dict(syms_cur)

+# This dictionary is used to pass a lookup table other than the system registry
+# to the parser. A global variable is necessary since the parser calls the
+# _find_var() and _find_func() functions below without extra arguments.
+_augmented_syms = {}

 #######################################################

@@ -2254,25 +2268,6 @@ def maxima_options(**kwds):
     """
     return ','.join(['%s=%s'%(key,mapped_opts(val)) for key, val in six.iteritems(kwds)])

-
-# Parser for symbolic ring elements
-
-# We keep two dictionaries syms_cur and syms_default to keep the current symbol
-# table and the state of the table at startup respectively. These are used by
-# the restore() function (see sage.misc.reset).
-#
-# The dictionary _syms is used as a lookup table for the system function
-# registry by _find_func() below. It gets updated by
-# symbolic_expression_from_string() before calling the parser.
-_syms = syms_cur = symbol_table.get('functions', {})
-syms_default = dict(syms_cur)
-
-# This dictionary is used to pass a lookup table other than the system registry
-# to the parser. A global variable is necessary since the parser calls the
-# _find_var() and _find_func() functions below without extra arguments.
-_augmented_syms = {}
-
-
 def _find_var(name):
     """
     Function to pass to Parser for constructing

[jhpalmieri]

comment:34

No, never mind, everything is fine.

[vbraun]

Changed branch from public/python3.6.1 to 7c393d7