Unable to use it for large graphs

[Remi-Boutin]

Hello, I have encountered the same error several times and I finally took some time to check that it indeed comes from the package. Here is a reproducible example :

import numpy
from netgraph import Graph
n_nodes = 2730
A = np.random.binomial(n =1, p = 0.002, size = (n_nodes ,n_nodes ))
clusters = np.random.randint(5, size=n_nodes )
G = nx.from_numpy_array(A, create_using=nx.DiGraph)
print('Is the graph weakly connected ? : {}'.format(nx.is_weakly_connected(G)))
print('Is the graph strongly connected ? : {}'.format(nx.is_weakly_connected(G)))
G_netgraph = Graph(G;
             node_layout="community",
             edge_layout='straight',
             node_layout_kwargs=dict(node_to_community = {node:clusters[node] for node in G.nodes} )
             )

I get the following error : KeyError: 13 The key changes with the graphs.

If you have a hint of what's happening, I would really appreciate it. I would love to use the package on large graph. The plots are really beautiful. Thanks Rémi

[paulbrodersen]

Hi Rémi, thanks for raising the issue and a very helpful MWE.

The issue occurred if a node was connected to other nodes but not to any nodes within the same community.
This violates the standard notion that communities in networks are subgraphs that are more densely connected than expected given the density in the rest of the graph.

I had implemented a workaround when fixing issue #41 but evidently I didn't cover all cases.

In the new version, nodes should be grouped with other nodes from the same community even if they don't connect to any of them -- as long as they belong to the same component. However, distinct components will still be plotted separately (as before) such that if nodes belong to the same community but different components, they will not be placed together (unless by chance).

The latter is a limitation of the Fruchterman-Reingold layout algorithm, which is the at the heart of my implementation of the community layout but which requires a single giant connected component. You can circumvent this limitation by adding additional edges to reconnect components. You can then either hide the edges (e.g. by setting their colour to white), or you can pre-compute the node positions using the augmented graph and then plot the original graph using the pre-computed positions.

In my experience, adding connections requires some experimentation to get visually pleasing results, so this is not something I am wanting to implement in netgraph at this point.

I would love to use the package on large graph.

Keep in mind that each node and each edge will require several 100 pixels to be discernible in the final plot (the artists themselves plus some whitespace around them). A monitor has about 2 million pixels available. You may easily run out of pixels if you try to plot the whole network in one figure.

---

Aside

You can use triple backticks (`) on github to denote a code block.

from netgraph import Graph

If you add the correct language tag after the first set of backticks, then github will even do some basic syntax highlighting.

```python
from netgraph import Graph

[paulbrodersen]

I will close the issue for now as your MWE now passes without any errors but feel free to re-open it if you run into further problems (or raise a new issue -- as appropriate).

[Remi-Boutin]

Hello Paul,

Thank you for your very informative answer ! I will keep in mind your advices regarding the plot of large graphs and the physical limitations of the screen. In my case, I want to plot large graphs to stress that my model renders more understandable results, so in a way it helps my point.

Also, Thank you for the kind github advices. As you probably guessed, I'm not very familiar with using it yet.

I'll continue to forward your package to colleagues asking for nice graph visualization tools in Python :-)

Have a good day.

[paulbrodersen]

Thank you for your kind words. You have a good day, too!