nix-apollo commented 4 months ago

Modularity Analysis

Description

Adds functionality for analyzing the modular structure of a RIB graphs. In particular, adds RIBGraph, a class that wraps around a networkit graph. This handles the conversion between networkits's numeric IDs and a Node class, as well as various plotting and analysis utilities.

Also adds rib_scripts/modularity/run_modularity.py. This script will run bisect edge ablations if it can't find them already, and then run the clustering algorithm using the thresholds from the edge ablations. It produces and saves several relevant plots along the way. This has two goals:

a low-friction entry point to running a suite of experiments you often want to run.
an example of how to chain the functions together that serve as a starting point for more customized analysis.

You are encouraged to make small PRs in the future making the script more customizable or adjusting the defaults.

Related Issue

Motivation and Context

This is trying to standardize an interface for modularity analysis that both works on small graphs and scales to pythia-1.4B.

How Has This Been Tested?

I added a small test suite for RIBGraph. Let me know if you think more tests are missing.

Does this PR introduce a breaking change?

No.

stefan-apollo commented 4 months ago

I think all the code we added to plot_graph.py should actually just be in plotting.py, plot_graph is just a run-script. And I don't think it's good that we import stuff from plot_graph elsewhere (run_modularity) now.

Not sure if it's worth the hassle though

stefan-apollo commented 4 months ago

Fine to merge once you added the docstring for clusters. (Plus fix tests, not sure why they're failing)

Would be happiest if you could move all the new code from plot_graph.py to plotting.py but if you disagree that's fine.

ApolloResearch / rib