search

sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.3k stars 447 forks source link

libGAP messes with Python subprocesses #14323

Closed jdemeyer closed 11 years ago

jdemeyer commented 11 years ago

I haven't figured out when/why, but this might happen:

sage -t devel/sage/sage/homology/simplicial_complex.py
**********************************************************************
File "devel/sage/sage/homology/simplicial_complex.py", line 3134, in sage.homology.simplicial_complex.SimplicialComplex.automorphism_group
Failed example:
    S.automorphism_group().is_isomorphic(SymmetricGroup(4))
Exception raised:
    Traceback (most recent call last):
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 455, in _run
        self.execute(example, compiled, test.globs)
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 811, in execute
        exec compiled in globs
      File "<doctest sage.homology.simplicial_complex.SimplicialComplex.automorphism_group[1]>", line 1, in <module>
        S.automorphism_group().is_isomorphic(SymmetricGroup(Integer(4)))
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/homology/simplicial_complex.py", line 3155, in automorphism_group
        [f.tuple() for f in self.facets()]])
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/graphs/generic_graph.py", line 16414, in automorphism_group
        output.append(PermutationGroup([perm_group_elt(aa) for aa in a]))
      File "refinement_graphs.pyx", line 941, in sage.groups.perm_gps.partn_ref.refinement_graphs.perm_group_elt (sage/groups/perm_gps/partn_ref/refinement_graphs.c:32796)
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/groups/perm_gps/permgroup.py", line 653, in __call__
        return self._element_class()(x, self, check=check)
      File "permgroup_element.pyx", line 452, in sage.groups.perm_gps.permgroup_element.PermutationGroupElement.__init__ (sage/groups/perm_gps/permgroup_element.c:4490)
      File "sage_object.pyx", line 474, in sage.structure.sage_object.SageObject._gap_ (sage/structure/sage_object.c:4544)
      File "sage_object.pyx", line 450, in sage.structure.sage_object.SageObject._interface_ (sage/structure/sage_object.c:4144)
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/interface.py", line 199, in __call__
        return cls(self, x, name=name)
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/expect.py", line 1280, in __init__
        self._name = parent._create(value, name=name)
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/interface.py", line 389, in _create
        self.set(name, value)
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/gap.py", line 1322, in set
        out = self._eval_line(cmd, allow_use_file=True)
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/gap.py", line 730, in _eval_line
        self._start()
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/gap.py", line 1170, in _start
        gap_reset_workspace(verbose=False)
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/gap.py", line 1496, in gap_reset_workspace
        g = Gap(use_workspace_cache=False, max_workspace_size=None)
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/gap.py", line 1084, in __init__
        max_workspace_size = _get_gap_memory_pool_size_MB()
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/gap.py", line 292, in _get_gap_memory_pool_size_MB
        pool = get_gap_memory_pool_size()
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/interfaces/gap.py", line 268, in get_gap_memory_pool_size
        suggested_size = min(suggested_size, int(mem.virtual_memory_limit()/10))
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python2.7/site-packages/sage/misc/memory_info.py", line 131, in virtual_memory_limit
        if platform.architecture()[0] == '32bit':
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python/platform.py", line 1084, in architecture
        output = _syscmd_file(executable, '')
      File "/padic/release/merger/sage-5.9.beta0/local/lib/python/platform.py", line 1032, in _syscmd_file
        rc = f.close()
    IOError: [Errno 10] No child processes
**********************************************************************

Looking at a strace, this is because libGAP installs a SIGCHLD signal handler which interferes with Python's multiprocessing.

spkg: http://boxen.math.washington.edu/home/jdemeyer/spkg/libgap-4.5.7.p2.spkg

(signal handler fix only, see https://bitbucket.org/vbraun/libgap/commits/3f4ebd64fc1219858802173cd86b7910040fef36)

CC: @vbraun @nexttime @roed314

Component: interfaces

Keywords: child processes

Author: Volker Braun

Reviewer: Jeroen Demeyer

Merged: sage-5.9.beta2

Issue created by migration from https://trac.sagemath.org/ticket/14323

jdemeyer commented 11 years ago
comment:2

Does Sage support libGAP child processes? Otherwise, that feature could simply be removed from libGAP.

83660e46-0051-498b-a8c1-f7a7bd232b5a commented 11 years ago

Changed keywords from none to child processes

vbraun commented 11 years ago
comment:4

I removed the SIGCHLD handler already, you just need to review #14039.

Close this ticket as duplicate.

vbraun commented 11 years ago

Author: Volker Braun

jdemeyer commented 11 years ago
comment:5

Replying to @vbraun:

Close this ticket as duplicate.

Not really. I'd like to fix this as soon as possible, which means without waiting for #14187+#14014+#14039.

vbraun commented 11 years ago
comment:6

Replying to @jdemeyer:

Not really. I'd like to fix this as soon as possible, which means without waiting for #14187+#14014+#14039.

Well tough, I would like those tickets reviewed as soon as possible because its an immense maintenance burden for me to carry around a patch that rebases all matrix groups.

jdemeyer commented 11 years ago

Reviewer: Jeroen Demeyer

jdemeyer commented 11 years ago

Description changed:

--- 
+++ 
@@ -54,3 +54,7 @@
 **********************************************************************

Looking at a strace, this is because libGAP installs a SIGCHLD signal handler which interferes with Python's multiprocessing. + +spkg (originally on #14014): http://www.stp.dias.ie/~vbraun/Sage/spkg/libgap-4.5.7.p2.spkg + +(signal handler fix only, see https://bitbucket.org/vbraun/libgap/commits/3f4ebd64fc1219858802173cd86b7910040fef36)

jdemeyer commented 11 years ago

Work Issues: SPKG.txt

jdemeyer commented 11 years ago
comment:8

Volker: you are missing an entry in SPKG.txt for this version. Should I add it manually or will you do it using your scripts?

vbraun commented 11 years ago
comment:9

I've added the SPKG entry in http://www.stp.dias.ie/~vbraun/Sage/spkg/libgap-4.6.2.p0.spkg

jdemeyer commented 11 years ago
comment:10

Replying to @vbraun:

I've added the SPKG entry in http://www.stp.dias.ie/~vbraun/Sage/spkg/libgap-4.6.2.p0.spkg

...which is the wrong version. This ticket is about libgap-4.5.7.p2.

vbraun commented 11 years ago
comment:11

Then I suggest we close this one as superseded by #14039.

jdemeyer commented 11 years ago
comment:12

Volker, I don't understand what you're trying to accomplish here. Acting like you're doing now is not helping. As I already said, this is an important issue which cannot wait for #14039.

I can totally understand your frustration that nobody is reviewing #14014, but this ticket shouldn't suffer from that. If you want reviewers for #14014, you could either ask David Roe (which is listed as reviewer) or ask sage-devel.

vbraun commented 11 years ago
comment:13

I don't understand why it is an issue to fix this blocker before sage-5.9 is released. We are currently at beta0, so it seems there will be at least 1-2 months until this needs to be fixed.

vbraun commented 11 years ago
comment:14

Sorry, wanted to say: I don't understand why it is an issue to fix this blocker long before sage-5.9 is released. We are currently at beta0, so it seems there will be at least 1-2 months until this needs to be fixed. If #14039 isn't reviewed by rc0 we can talk.

jdemeyer commented 11 years ago
comment:15

Why wait for rc0 to fix this? We know the problem and we know the fix.

Imagine we don't fix this before sage-5.9.rc0. The problem on this ticket is causing random strange failures, so for every multiprocessing-related problem that people report, we would need to check whether this ticket fixes it.

vbraun commented 11 years ago
comment:16

Replying to @jdemeyer:

Why wait for rc0 to fix this? We know the problem and we know the fix.

And the fix is at #14039.

Imagine we don't fix this before sage-5.9.rc0. The problem on this ticket is causing random strange failures, so for every multiprocessing-related problem that people report, we would need to check whether this ticket fixes it.

I'll be happy to tell every one of them that this is their own fault for not reviewing #14039.

Maybe David can tell us how long it'll take...

jdemeyer commented 11 years ago

Changed work issues from SPKG.txt to none

jdemeyer commented 11 years ago
comment:17

Fixed SPKG.txt.

jdemeyer commented 11 years ago

Description changed:

--- 
+++ 
@@ -55,6 +55,6 @@

Looking at a strace, this is because libGAP installs a SIGCHLD signal handler which interferes with Python's multiprocessing.

-spkg (originally on #14014): http://www.stp.dias.ie/~vbraun/Sage/spkg/libgap-4.5.7.p2.spkg +spkg: http://boxen.math.washington.edu/home/jdemeyer/spkg/libgap-4.5.7.p2.spkg

(signal handler fix only, see https://bitbucket.org/vbraun/libgap/commits/3f4ebd64fc1219858802173cd86b7910040fef36)

83660e46-0051-498b-a8c1-f7a7bd232b5a commented 11 years ago
comment:18

FWIW, I've built and tested Sage 5.9.beta0 from scratch with this spkg without issues, but I didn't have problems before either. (Ubuntu 10.04.4 LTS x86_64)

jdemeyer commented 11 years ago

Merged: sage-5.9.beta2