barahona-research-group / PyGenStability

PyGenStability: Multiscale community detection with generalized Markov Stability
https://barahona-research-group.github.io/PyGenStability/
GNU General Public License v3.0
32 stars 11 forks source link

scale selection at margins of domain #62

Closed d-schindler closed 1 year ago

d-schindler commented 1 year ago

The current scale selection criterion is not reliable at the margins of the domain of Markov scale t due to:

  1. Cut-off through windowing
  2. The diagonal count of low values in the NVI(t,t') necessarily leads to smaller values at the margins because the diagonal is smaller. Hence, 1-NVI(t,.) tends to produce anomalous local minima at the margins.

While 1) is perhaps acceptable, Mauricio and me discussed that we should come up with a solution for 2). I will think about this and try a couple of things.

arnaudon commented 1 year ago

Can you somehow normalize by diagonal size?

arnaudon commented 1 year ago

Or just don't compute it at the margin, and write in docsctring why it is not reliable, so we don't compute it. User can increase markov time min/max to have larger range of scale detection.

d-schindler commented 1 year ago

I developed a new algorithm with @mauriciobarahona that seems to work better. The idea is to pool the NVI(t,t') matrix and look at the local minima of its diagonal to identify the block structures. I will create a new branch for my implementation once we merge the Leiden implementation. This new scale selection algorithm will need a slightly modified summary plot, I will try to adapt. Hopefully, I will be quick to update everything for our paper submission and not cause any delays.

arnaudon commented 1 year ago

Excellent!