sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.44k stars 481 forks source link

Permanently get rid of bare except: statements #32788

Closed edd8e884-f507-429a-b577-5d554626c0fe closed 2 years ago

edd8e884-f507-429a-b577-5d554626c0fe commented 3 years ago

A while ago, @jdemeyer used to regularly take care of removing bare except: statements constantly introduced in Sage source code, se e.g. #27427, #14028, #11310, #21687, #24274.

The goal of this ticket (perhaps should it be a task ticket ?) is to take over the work without self-abnegation by:

CC: @kliem @frederichan-IMJPRG @videlec

Component: misc

Author: Frédéric Chapoton, Jonathan Kliem, Thierry Monteil

Branch/Commit: de4faf6

Reviewer: Jonathan Kliem, Markus Wageringel

Issue created by migration from https://trac.sagemath.org/ticket/32788

kliem commented 3 years ago
comment:37

Well, I would surely be interested in that bug. Looks like it's my code causing the problems...

edd8e884-f507-429a-b577-5d554626c0fe commented 3 years ago
comment:38

I made a distclean, but the logs are still available on the tickets, let us see if it reappears.

edd8e884-f507-429a-b577-5d554626c0fe commented 3 years ago
comment:39

Regarding src/sage/geometry/polyhedron/base.py there are serious doctesting issues in 32bit architectures as well.

fchapoton commented 3 years ago
comment:40

there is at least one green patchbot. Good to go ?

mkoeppe commented 3 years ago
comment:41
./sage -tp --optional=sage,polymake,jupymake src/sage/interfaces/polymake.py src/sage/geometry/polyhedron/

sage -t --random-seed=193726426221037022828737471270892163828 src/sage/interfaces/polymake.py
**********************************************************************
File "src/sage/interfaces/polymake.py", line 1534, in sage.interfaces.polymake.PolymakeElement._sage_
Failed example:
    PP.F_VECTOR.sage()      # optional - polymake
Exception raised:
    Traceback (most recent call last):
      File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/lib/python3.9/site-packages/sage/doctest/forker.py", line 694, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/lib/python3.9/site-packages/sage/doctest/forker.py", line 1088, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.interfaces.polymake.PolymakeElement._sage_[11]>", line 1, in <module>
        PP.F_VECTOR.sage()      # optional - polymake
      File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/lib/python3.9/site-packages/sage/interfaces/interface.py", line 1106, in sage
        return self._sage_(*args, **kwds)
      File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/lib/python3.9/site-packages/sage/interfaces/polymake.py", line 1587, in _sage_
        return vector(base_ring, [str_to_base_ring(s)
    UnboundLocalError: local variable 'base_ring' referenced before assignment

File "src/sage/interfaces/polymake.py", line 1551, in sage.interfaces.polymake.PolymakeElement._sage_
Failed example:
    polymake.icosahedron().sage() # optional - polymake
Exception raised:
    Traceback (most recent call last):
      File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/lib/python3.9/site-packages/sage/doctest/forker.py", line 694, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/lib/python3.9/site-packages/sage/doctest/forker.py", line 1088, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.interfaces.polymake.PolymakeElement._sage_[16]>", line 1, in <module>
        polymake.icosahedron().sage() # optional - polymake
      File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/lib/python3.9/site-packages/sage/interfaces/interface.py", line 1106, in sage
        return self._sage_(*args, **kwds)
      File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/lib/python3.9/site-packages/sage/interfaces/polymake.py", line 1569, in _sage_
        d = int(r[i + 1: i + i1])
    ValueError: invalid literal for int() with base 10: 'aticExtension<Rational>>[SAGE2]'
fchapoton commented 3 years ago
comment:42

I propose, as a strategy, to undo the changes in polymake interface and keep them for another ticket. This would also mean not to activate E722 here, but later.

Otherwise, somebody should stand up and fix now the polymake issues. I will not.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

30b8dcamark polymake_expect import as optional
40b8827two fixes in polymake type conversion
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Changed commit from a903f72 to 40b8827

kliem commented 3 years ago
comment:44

There first change was necessary, because it changed polymake to polymake_expect, which does not work for me at all.

I still get one failure and I'm really confused, that I get this and you don't:

sage -t --random-seed=226050575114945871045540898771575845983 src/sage/interfaces/polymake.py
**********************************************************************
File "src/sage/interfaces/polymake.py", line 1544, in sage.interfaces.polymake.PolymakeElement._sage_
Failed example:
    _.parent()              # optional - polymake
Expected:
    Full MatrixSpace of 2 by 2 dense matrices over Integer Ring
Got:
    Full MatrixSpace of 2 by 2 dense matrices over Rational Field
**********************************************************************
1 item had failures:
   1 of  18 in sage.interfaces.polymake.PolymakeElement._sage_
    [282 tests, 1 failure, 3.17 s]
mkoeppe commented 3 years ago
comment:45

The changes to PolymakeElement look nontrivial, so I would agree with the suggestion to defer them to a separate ticket.

mkoeppe commented 3 years ago
comment:46

... or just try the minimal change that changes except: to except Exception:

fchapoton commented 3 years ago

Changed branch from public/ticket/32788 to u/chapoton/except_fix_branch

fchapoton commented 3 years ago
comment:47

I have made a branch with only a minimal change to polymake file. And squashed all the commits into one single commit.


New commits:

1afd3a3fix all E722 and activate the check
fchapoton commented 3 years ago

Changed commit from 40b8827 to 1afd3a3

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Changed commit from 1afd3a3 to de356d6

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 3 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

de356d6forgotten one
mkoeppe commented 3 years ago
comment:49

Polymake works OK now with this branch, thanks.

fchapoton commented 2 years ago
comment:50

green bot, can we move on here, please ?

kliem commented 2 years ago
comment:51

Replying to @sagetrac-git:

Branch pushed to git repo; I updated commit sha1. New commits:

30b8dcamark polymake_expect import as optional

Can we please keep this commit. Pexpect does not work in my case and importing polymake_expect as polymake is not a good idea in that case.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

2ec7374add one tag in polymake interface
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from de356d6 to 2ec7374

fchapoton commented 2 years ago
comment:53

ok, done

kliem commented 2 years ago
comment:54

I think the comment should move before the code and not after:

-                rat_points.add(X(point)) # checks if this point lies on X or not
-            except:
+                rat_points.add(X(point))
+                # checks if this point lies on X or not

So instead we should have something as

                # checks if this point lies on X or not
                rat_points.add(X(point))
kliem commented 2 years ago

Reviewer: Jonathan Kliem

edd8e884-f507-429a-b577-5d554626c0fe commented 2 years ago
comment:56

Replying to @fchapoton:

I have made a branch with only a minimal change to polymake file. And squashed all the commits into one single commit.

I do not see the benefit of such squashing, but i see the drawbacks: we get a mixed bomb of changes without understanding which change is done for which reason. For example, i split the changes on giac.pyx as follows in order to ease reviewing:

​bcdb620    #32788 : fix intentations for libs/giac/giac.pyx
​da3e437    #32788 : remove useless try/except statements in src/sage/libs/giac/giac.pyx

Similarly, it is important to have separate commits when you rename a file and modify it.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

ec236eccomment before
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from 2ec7374 to ec236ec

fchapoton commented 2 years ago
comment:58

comment:54 is now fixed

concerning squashing, I thought it was a good idea to simplify the commit history. I agree this was maybe not a so good idea. If you think this matters enough to spend your time on that, please provide a better branch

fchapoton commented 2 years ago
comment:59

green bot again

fchapoton commented 2 years ago
comment:60

can we please move on here ?

edd8e884-f507-429a-b577-5d554626c0fe commented 2 years ago

Changed branch from u/chapoton/except_fix_branch to u/tmonteil/except_fix_branch

edd8e884-f507-429a-b577-5d554626c0fe commented 2 years ago

Changed commit from ec236ec to 9522357

edd8e884-f507-429a-b577-5d554626c0fe commented 2 years ago
comment:62

I cherry-picked the relevant commits and merged with 9.5.beta6 so that git diff trac/u/chapoton/except_fix_branch trac/u/tmonteil/except_fix_branch leads to:

--- a/src/sage/interfaces/polymake.py
+++ b/src/sage/interfaces/polymake.py
@@ -364,7 +364,7 @@ class PolymakeAbstract(ExtraTabCompletion, Interface):
             try:
                 x = RDF(x)
                 return '{}'.format(x)
-            except Exception:
+            except (TypeError, ValueError):
                 pass

             raise NotImplementedError

I am not sure if i have to revert that change.


Last 10 new commits:

be872c1some fixes in giac, schemes, geometry
35cab30Merge branch 'u/chapoton/32788' of git://trac.sagemath.org/sage into u/gh-kliem/32788
0113b72code style again
6c08ce1remove blank except in GIAC call
04a1f01fix distinction between two calls
9905930remove code duplication
d3df701#32788 : ensure bare except: statements will not reappear
e92b905mark polymake_expect import as optional
f463e49comment before
9522357Merge branch 'develop.9.5.beta6' into HEAD
mkoeppe commented 2 years ago
comment:63

Stalled in needs_review or needs_info; likely won't make it into Sage 9.5.

fchapoton commented 2 years ago
comment:64

red branch => needs work

fchapoton commented 2 years ago

Changed branch from u/tmonteil/except_fix_branch to public/except_fix_branch

fchapoton commented 2 years ago

Changed commit from 9522357 to db57c4f

fchapoton commented 2 years ago
comment:65

rebased on 9.5.rc1


New commits:

58865cevarious lgtm-suggested details
db57c4fMerge branch 'u/tmonteil/except_fix_branch' in 9.5.rc1
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from db57c4f to de4faf6

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

de4faf6Merge branch 'u/tmonteil/except_fix_branch' in 9.5.rc1
fchapoton commented 2 years ago
comment:67

now correctly rebased, needs test by somebody having polymake

mwageringel commented 2 years ago
comment:68

There are two polymake test failures (see below), but they are not related to this ticket, so are best dealt with in another ticket.

I am ready to set this ticket to positive, if someone could confirm that the new pycodestyle check still passes with 9.5 merged. Unfortunately, I have difficulties in running tox on my machine.

By the way, this ticket will also fix a random doctest failure in coding/linear_code.py which is due to an AlarmInterrupt being swallowed by a bare except.


sage -t --long --warn-long 69.2 --random-seed=17721019333867817533235853689613643341 src/doc/en/thematic_tutorials/geometry/polyhedra_tutorial.rst
**********************************************************************
File "src/doc/en/thematic_tutorials/geometry/polyhedra_tutorial.rst", line 650, in doc.en.thematic_tutorials.geometry.polyhedra_tutorial
Failed example:
    Polyhedron(vertices=V, backend='polymake')               # optional - polymake
Expected:
    A 3-dimensional polyhedron in (Number Field in sqrt5 with defining polynomial x^2 - 5)^3 defined as the convex hull of 20 vertices
Got:
    A 3-dimensional polyhedron in (Number Field in sqrt5 with defining polynomial x^2 - 5 with sqrt5 = 2.236067977499790?)^3 defined as the convex hull of 20 v
ertices

sage -t --long --warn-long 69.2 --random-seed=201256750698610760769760302766889916051 src/sage/interfaces/polymake.py
**********************************************************************
File "src/sage/interfaces/polymake.py", line 2189, in sage.interfaces.polymake.PolymakeExpect._eval_line
Failed example:
    c.N_VERTICES                      # optional - polymake_expect
Expected:
    32768
Got:
    Can't locate object method "description" via package "32768" (perhaps you forgot to load "32768"?) at input line 1.
mwageringel commented 2 years ago

Changed reviewer from Jonathan Kliem to Jonathan Kliem, Markus Wageringel

mwageringel commented 2 years ago
comment:69

Never mind. I have tried it on a different machine and pycodestyle-minimal passes, so I think this is good to go.

slel commented 2 years ago
comment:70

In src/sage/libs/giac/giac.pyx, docstring indentation for loadgiacgen seems off.

mwageringel commented 2 years ago
comment:71

Replying to @slel:

In src/sage/libs/giac/giac.pyx, docstring indentation for loadgiacgen seems off.

I am happy to review it if anyone wants to change that, but I do not think it is necessary here.

vbraun commented 2 years ago

Changed branch from public/except_fix_branch to de4faf6