Segmentation fault running ModularityVertexPartition

[jeshi96]

Hi,

I'm encountering a segmentation fault when running ModularityVertexPartition on the DBLP graph (https://snap.stanford.edu/data/com-DBLP.html). I'm not entirely sure if my install is correct (specifically the igraph install), but there seem to be no other errors that arise. I installed leidenalg using pip, and I'm testing on a GCP instance running Ubuntu, 16.04 LTS, amd64 (60 vCPUs, 240 GiB memory).

The Python code that I'm running is as follows:

import leidenalg
from igraph import *

def main():
  g = Graph.Read_Ncol('/home/jeshi/snap/dblp.edges', directed=False)
  part = leidenalg.find_partition(g, leidenalg.ModularityVertexPartition);

if __name__=="__main__":
  main()

Note that the dblp.edges graph is exactly the graph downloaded from SNAP, but with the first four commented lines removed using sed.

Thanks for your help!

[vtraag]

I can't reproduce the problem locally (Ubuntu 20.04, Python 3.6.10), using the same DBLP graph. Perhaps something is going wrong with the installation of igraph instead of leidenalg? Could you perhaps report the output of g.summary(), just to check if that is going well?

One other possibly relevant aspect is the versions of igraph and python-igraph, what versions are you running?

[jeshi96]

The output of g.summary() seems fine (this is for the amazon graph, but it also works for dblp):

IGRAPH UN-- 334863 925872 -- 

I checked the versions for igraph -- the python-igraph version is 0.8.2, and the C core version is 0.8.2 as well.

[vtraag]

Hmm, OK, that sounds difficult to debug.

Do you only encounter a segmentation fault when using this particular graph? Or does it happen with all graphs? Perhaps you can first try with an example graph (e.g. Graph.Famous('Zachary')).

Do you have access to the core dump, and could you possibly attach that?

[jeshi96]

Unfortunately, it's the same error using Graph.Famous('Zachary'). I tried running it with gdb; here's the stack trace, if it's helpful:

Program received signal SIGSEGV, Segmentation fault.
Optimiser::optimise_partition (this=this@entry=0xc69160, partitions=std::vector of length 1, capacity 1 = {...}, layer_weights=std::vector of length 1, capacity 1 = {...}, 
    is_membership_fixed=std::vector<bool> of length 34, capacity 64 = {...}, max_comm_size=max_comm_size@entry=0) at src/Optimiser.cpp:110
110     if (graph->vcount() != n)
(gdb) backtrace
#0  Optimiser::optimise_partition (this=this@entry=0xc69160, partitions=std::vector of length 1, capacity 1 = {...}, layer_weights=std::vector of length 1, capacity 1 = {...}, 
    is_membership_fixed=std::vector<bool> of length 34, capacity 64 = {...}, max_comm_size=max_comm_size@entry=0) at src/Optimiser.cpp:110
#1  0x00007ffff4adcac6 in Optimiser::optimise_partition (this=this@entry=0xc69160, partition=partition@entry=0xbe58d0, 
    is_membership_fixed=std::vector<bool> of length 34, capacity 64 = {...}, max_comm_size=0) at src/Optimiser.cpp:68
#2  0x00007ffff4adcb79 in Optimiser::optimise_partition (this=this@entry=0xc69160, partition=partition@entry=0xbe58d0, 
    is_membership_fixed=std::vector<bool> of length 34, capacity 64 = {...}) at src/Optimiser.cpp:60
#3  0x00007ffff4ae252c in _Optimiser_optimise_partition (self=<optimized out>, args=<optimized out>, keywds=<optimized out>) at src/python_optimiser_interface.cpp:99
#4  0x00000000004c4700 in do_call (nk=<optimized out>, na=<optimized out>, pp_stack=0x7fffffffdc80, func=<optimized out>) at ../Python/ceval.c:4564
#5  call_function (oparg=<optimized out>, pp_stack=0x7fffffffdc80) at ../Python/ceval.c:4372
#6  PyEval_EvalFrameEx (f=
    Frame 0xe07b00, for file /home/jeshi/.local/lib/python2.7/site-packages/leidenalg/Optimiser.py, line 296, in optimise_partition (self=<Optimiser(_optimiser=<PyCapsule at remote 0x7ffff4a94300>) at remote 0x7ffff4e11510>, partition=<ModularityVertexPartition(_membership=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33], _partition=<PyCapsule at remote 0x7ffff4a942a0>, _graph=<Graph at remote 0x7ffff4e0c338>, _modularity=None, _len=34, _modularity_dirty=True, _modularity_params={}) at remote 0x7ffff44bb550>, n_iterations=2, is_membership_fixed=None, itr=0, diff=0, continue_iteration=True), throwflag=<optimized out>) at ../Python/ceval.c:2987
#7  0x00000000004ba306 in PyEval_EvalCodeEx (co=0x7ffff4e02db0, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kws=<optimized out>, 
    kwcount=0, defs=0x7ffff4e32890, defcount=2, closure=0x0) at ../Python/ceval.c:3582
#8  0x00000000004c1d82 in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffde70, func=<optimized out>) at ../Python/ceval.c:4445
#9  call_function (oparg=<optimized out>, pp_stack=0x7fffffffde70) at ../Python/ceval.c:4370
#10 PyEval_EvalFrameEx (
    f=Frame 0xd96590, for file /home/jeshi/.local/lib/python2.7/site-packages/leidenalg/functions.py, line 96, in find_partition (graph=<Graph at remote 0x7ffff4e0c338>, partition_type=<type at remote 0xb957f0>, initial_membership=None, weights=None, n_iterations=2, max_comm_size=0, seed=None, kwargs={}, partition=<ModularityVertexPartition(_membership=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33], _partition=<PyCapsule at remote 0x7ffff4a942a0>, _graph=<...>, _modularity=None, _len=34, _modularity_dirty=True, _modularity_params={}) at remote 0x7ffff44bb550>, optimiser=<Optimiser(_optimiser=<PyCapsule at remote 0x7ffff4a94300>) at remote 0x7ffff4e11510>), throwflag=<optimized out>) at ../Python/ceval.c:2987
#11 0x00000000004ba306 in PyEval_EvalCodeEx (co=0x7ffff7ed0830, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kws=<optimized out>, 
    kwcount=0, defs=0x7ffff7e0f6c8, defcount=5, closure=0x0) at ../Python/ceval.c:3582
#12 0x00000000004c1d82 in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffe060, func=<optimized out>) at ../Python/ceval.c:4445
#13 call_function (oparg=<optimized out>, pp_stack=0x7fffffffe060) at ../Python/ceval.c:4370
#14 PyEval_EvalFrameEx (f=Frame 0xd93830, for file test_mod.py, line 10, in main (g=<Graph at remote 0x7ffff4e0c338>), throwflag=<optimized out>) at ../Python/ceval.c:2987
#15 0x00000000004c1874 in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffffffe170, func=<optimized out>) at ../Python/ceval.c:4435
#16 call_function (oparg=<optimized out>, pp_stack=0x7fffffffe170) at ../Python/ceval.c:4370
#17 PyEval_EvalFrameEx (f=Frame 0x7ffff7fd3c20, for file test_mod.py, line 13, in <module> (), throwflag=<optimized out>) at ../Python/ceval.c:2987
#18 0x00000000004ba306 in PyEval_EvalCodeEx (co=0x7ffff7ee6930, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kws=<optimized out>, 
    kwcount=0, defs=0x0, defcount=0, closure=0x0) at ../Python/ceval.c:3582
#19 0x00000000004eb42f in PyEval_EvalCode (
    locals={'DyadCensus': <type at remote 0xab99f0>, 'BLISS_FL': 1, 'REWIRING_SIMPLE_LOOPS': 1, 'izip': <type at remote 0x924e60>, 'ClusterColoringPalette': <type at remote 0xaf35a0>, 'Flow': <type at remote 0xb69fd0>, 'BLISS_FM': 3, 'Dendrogram': <type at remote 0xa97a50>, 'BLISS_FS': 2, 'hsl_to_rgb': <function at remote 0x7ffff540cb90>, 'cut': <module at remote 0x7ffff7e47830>, 'layout': <module at remote 0x7ffff4e8c478>, 'Vertex': <type at remote 0x7ffff6ca3560>, 'Histogram': <type at remote 0xb623d0>, 'GraphBase': <type at remote 0x7ffff6ca1940>, 'STAR_OUT': 0, 'STAR_UNDIRECTED': 2, 'save': <function at remote 0x7ffff4df6c80>, 'RainbowPalette': <type at remote 0xb0b670>, 'EdgeSeq': <type at remote 0xa75000>, 'read': <function at remote 0x7ffff4df6c08>, '__file__': 'test_mod.py', 'set_status_handler': <built-in function set_status_handler>, 'GET_ADJACENCY_UPPER': 0, 'name': 'write_svg', 'GraphSummary': <type at remote 0xb6b0c0>, 'utils': <module at remote 0x7ffff53f08d8>, 'rgba_to_hsla': <function at remote 0x7ffff54...(truncated), 
    globals={'DyadCensus': <type at remote 0xab99f0>, 'BLISS_FL': 1, 'REWIRING_SIMPLE_LOOPS': 1, 'izip': <type at remote 0x924e60>, 'ClusterColoringPalette': <type at remote 0xaf35a0>, 'Flow': <type at remote 0xb69fd0>, 'BLISS_FM': 3, 'Dendrogram': <type at remote 0xa97a50>, 'BLISS_FS': 2, 'hsl_to_rgb': <function at remote 0x7ffff540cb90>, 'cut': <module at remote 0x7ffff7e47830>, 'layout': <module at remote 0x7ffff4e8c478>, 'Vertex': <type at remote 0x7ffff6ca3560>, 'Histogram': <type at remote 0xb623d0>, 'GraphBase': <type at remote 0x7ffff6ca1940>, 'STAR_OUT': 0, 'STAR_UNDIRECTED': 2, 'save': <function at remote 0x7ffff4df6c80>, 'RainbowPalette': <type at remote 0xb0b670>, 'EdgeSeq': <type at remote 0xa75000>, 'read': <function at remote 0x7ffff4df6c08>, '__file__': 'test_mod.py', 'set_status_handler': <built-in function set_status_handler>, 'GET_ADJACENCY_UPPER': 0, 'name': 'write_svg', 'GraphSummary': <type at remote 0xb6b0c0>, 'utils': <module at remote 0x7ffff53f08d8>, 'rgba_to_hsla': <function at remote 0x7ffff54...(truncated), co=0x7ffff7ee6930) at ../Python/ceval.c:669
#20 run_mod.lto_priv.2012 (mod=<optimized out>, filename=<optimized out>, 
    globals={'DyadCensus': <type at remote 0xab99f0>, 'BLISS_FL': 1, 'REWIRING_SIMPLE_LOOPS': 1, 'izip': <type at remote 0x924e60>, 'ClusterColoringPalette': <type at remote 0xaf35a0>, 'Flow': <type at remote 0xb69fd0>, 'BLISS_FM': 3, 'Dendrogram': <type at remote 0xa97a50>, 'BLISS_FS': 2, 'hsl_to_rgb': <function at remote 0x7ffff540cb90>, 'cut': <module at remote 0x7ffff7e47830>, 'layout': <module at remote 0x7ffff4e8c478>, 'Vertex': <type at remote 0x7ffff6ca3560>, 'Histogram': <type at remote 0xb623d0>, 'GraphBase': <type at remote 0x7ffff6ca1940>, 'STAR_OUT': 0, 'STAR_UNDIRECTED': 2, 'save': <function at remote 0x7ffff4df6c80>, 'RainbowPalette': <type at remote 0xb0b670>, 'EdgeSeq': <type at remote 0xa75000>, 'read': <function at remote 0x7ffff4df6c08>, '__file__': 'test_mod.py', 'set_status_handler': <built-in function set_status_handler>, 'GET_ADJACENCY_UPPER': 0, 'name': 'write_svg', 'GraphSummary': <type at remote 0xb6b0c0>, 'utils': <module at remote 0x7ffff53f08d8>, 'rgba_to_hsla': <function at remote 0x7ffff54...(truncated), 
    locals={'DyadCensus': <type at remote 0xab99f0>, 'BLISS_FL': 1, 'REWIRING_SIMPLE_LOOPS': 1, 'izip': <type at remote 0x924e60>, 'ClusterColoringPalette': <type at remote 0xaf35a0>, 'Flow': <type at remote 0xb69fd0>, 'BLISS_FM': 3, 'Dendrogram': <type at remote 0xa97a50>, 'BLISS_FS': 2, 'hsl_to_rgb': <function at remote 0x7ffff540cb90>, 'cut': <module at remote 0x7ffff7e47830>, 'layout': <module at remote 0x7ffff4e8c478>, 'Vertex': <type at remote 0x7ffff6ca3560>, 'Histogram': <type at remote 0xb623d0>, 'GraphBase': <type at remote 0x7ffff6ca1940>, 'STAR_OUT': 0, 'STAR_UNDIRECTED': 2, 'save': <function at remote 0x7ffff4df6c80>, 'RainbowPalette': <type at remote 0xb0b670>, 'EdgeSeq': <type at remote 0xa75000>, 'read': <function at remote 0x7ffff4df6c08>, '__file__': 'test_mod.py', 'set_status_handler': <built-in function set_status_handler>, 'GET_ADJACENCY_UPPER': 0, 'name': 'write_svg', 'GraphSummary': <type --Type <RET> for more, q to quit, c to continue without paging--
at remote 0xb6b0c0>, 'utils': <module at remote 0x7ffff53f08d8>, 'rgba_to_hsla': <function at remote 0x7ffff54...(truncated), flags=<optimized out>, arena=<optimized out>)
    at ../Python/pythonrun.c:1376
#21 0x00000000004e56a2 in PyRun_FileExFlags (fp=0x9e54d0, filename=0x7fffffffe7f4 "test_mod.py", start=<optimized out>, 
    globals={'DyadCensus': <type at remote 0xab99f0>, 'BLISS_FL': 1, 'REWIRING_SIMPLE_LOOPS': 1, 'izip': <type at remote 0x924e60>, 'ClusterColoringPalette': <type at remote 0xaf35a0>, 'Flow': <type at remote 0xb69fd0>, 'BLISS_FM': 3, 'Dendrogram': <type at remote 0xa97a50>, 'BLISS_FS': 2, 'hsl_to_rgb': <function at remote 0x7ffff540cb90>, 'cut': <module at remote 0x7ffff7e47830>, 'layout': <module at remote 0x7ffff4e8c478>, 'Vertex': <type at remote 0x7ffff6ca3560>, 'Histogram': <type at remote 0xb623d0>, 'GraphBase': <type at remote 0x7ffff6ca1940>, 'STAR_OUT': 0, 'STAR_UNDIRECTED': 2, 'save': <function at remote 0x7ffff4df6c80>, 'RainbowPalette': <type at remote 0xb0b670>, 'EdgeSeq': <type at remote 0xa75000>, 'read': <function at remote 0x7ffff4df6c08>, '__file__': 'test_mod.py', 'set_status_handler': <built-in function set_status_handler>, 'GET_ADJACENCY_UPPER': 0, 'name': 'write_svg', 'GraphSummary': <type at remote 0xb6b0c0>, 'utils': <module at remote 0x7ffff53f08d8>, 'rgba_to_hsla': <function at remote 0x7ffff54...(truncated), 
    locals={'DyadCensus': <type at remote 0xab99f0>, 'BLISS_FL': 1, 'REWIRING_SIMPLE_LOOPS': 1, 'izip': <type at remote 0x924e60>, 'ClusterColoringPalette': <type at remote 0xaf35a0>, 'Flow': <type at remote 0xb69fd0>, 'BLISS_FM': 3, 'Dendrogram': <type at remote 0xa97a50>, 'BLISS_FS': 2, 'hsl_to_rgb': <function at remote 0x7ffff540cb90>, 'cut': <module at remote 0x7ffff7e47830>, 'layout': <module at remote 0x7ffff4e8c478>, 'Vertex': <type at remote 0x7ffff6ca3560>, 'Histogram': <type at remote 0xb623d0>, 'GraphBase': <type at remote 0x7ffff6ca1940>, 'STAR_OUT': 0, 'STAR_UNDIRECTED': 2, 'save': <function at remote 0x7ffff4df6c80>, 'RainbowPalette': <type at remote 0xb0b670>, 'EdgeSeq': <type at remote 0xa75000>, 'read': <function at remote 0x7ffff4df6c08>, '__file__': 'test_mod.py', 'set_status_handler': <built-in function set_status_handler>, 'GET_ADJACENCY_UPPER': 0, 'name': 'write_svg', 'GraphSummary': <type at remote 0xb6b0c0>, 'utils': <module at remote 0x7ffff53f08d8>, 'rgba_to_hsla': <function at remote 0x7ffff54...(truncated), closeit=1, flags=0x7fffffffe3f0)
    at ../Python/pythonrun.c:1362
#22 0x00000000004e3f56 in PyRun_SimpleFileExFlags (fp=0x9e54d0, filename=<optimized out>, closeit=1, flags=0x7fffffffe3f0) at ../Python/pythonrun.c:948
#23 0x0000000000493dee in Py_Main (argc=<optimized out>, argv=<optimized out>) at ../Modules/main.c:640
#24 0x00007ffff7810840 in __libc_start_main (main=0x493890 <main>, argc=2, argv=0x7fffffffe5b8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, 
    stack_end=0x7fffffffe5a8) at ../csu/libc-start.c:291
#25 0x00000000004937b9 in _start ()

[vtraag]

At least good that you get the same problem with other graphs, that makes it somewhat easier hopefully. Thanks for the backtrace, that is helpful.

The error itself is very strange, it's not clear what is going wrong. It seems as though the internal igraph object is not being properly intialised within the leidenalg package.

Could you perhaps run the following

opt = leidenalg.Optimiser()
part = leidenalg.ModularityVertexPartition(G)
part.diff_move(0, 1)

If you hit an error, a backtrace would be helpful again.

[jeshi96]

I tried running the code snippet on the Zachary graph (34 vertices, 78 edges) -- oddly enough, it doesn't give a segmentation fault, but it hasn't finished running yet. The last line part.diff_move(0, 1) has been running for the past ~10 minutes or so -- is it supposed to take this long?

[vtraag]

No, it should be a millisecond calculation. Do you have the igraph C library installed separately?

[jeshi96]

Yes, I do have the C library installed separately. I had initially tried installing leidenalg without pre-installing a C core, but the error I got then was that it could not find igraph.h.

[jeshi96]

I've just figured out the issue -- I was using Python 2 to run, instead of Python 3. Everything works now that I've switched to Python 3. Thanks!

[vtraag]

Ha, good to see you sorted it out! It should not ordinarily have the problem that is can't find igraph.h when compiling from source. If you install using pip a binary wheel should be provided by the way (but only for Python3 indeed).