Retain weights after `disparity()`

[baslat]

Hi, thanks for your work on the package. Would it be possible to allow the results of a call to disparity() to retain the weights? At the moment the resulting graph just has the nodes and edges, but it would be useful if the weights and any other data associated with nodes or edges were retained.

[zpneal]

Thanks for the suggestion. This comes up from time to time, but I've been reluctant to implement it.

Backbone models like disparity() and the other models in the package begin from the assumption that the edge weights in a given network cannot be interpreted as having a single scale, and therefore cannot be pruned using a single threshold value. Instead, proper interpretation of edge weights requires considering a null model. A weight that is "large" for one edge may be "small" for a different edge. This means that the edge weights themselves are not particularly informative, and should not be used for analysis after extracting a backbone. I'm working on a paper now that gets into the details of the issue, and provides some examples. I'd be happy to share a copy when it's ready.

If you did have a case where interpreting the weights of retained edges was sensible, this isn't too difficult using the existing package. For example, if M is your weighted adjacency matrix, you could use:

backbone <- disparity(M)
weighted_backbone <- M * backbone

As for node attributes, these should be retained by disparity() as long as the starting object was an igraph object with node attributes. If this isn't working for you, it's certainly a bug needs to be fixed. Let me know.

[baslat]

I understand what you're staying. I'd be interested to see the paper once ready.

My typical use case for wanting to retain edge weights is for visualisation of the backbone. I find it useful to map the alpha value of the edge visualisation to the edge weight so that I can still get a quick sense of where the most "volume" is in the backbone.

You're correct in that node attributes are retained. Edge attributes are lost however. Sometimes I have set edge attributes for visualisation (eg with visNetwork), and sometimes for data about the edge itself.

I get an error when trying your suggestion to multiply an adjacency matrix and backbone (below), however I was able to refactor this to get it to work for me.

> bb <- disparity(g)
> M <- as_adjacency_matrix(g, attr = "weight", sparse=FALSE)
> weighted_backbone <- M * bb
Error in `*.igraph`:
! Cannot multiply igraph graph with this type
Hide Traceback
    ▆
 1. └─igraph:::`*.igraph`(M, bb)

[zpneal]

I've just finished a draft of the paper. You can find a copy (with the code) at https://osf.io/u8m7n/.

In the paper, I frame the issue as coming down to the question: "Does an edge with a weight of w have the same meaning as another edge with a weight of w in the same network?" If it does, then it makes sense to use the weights in analysis and visualization because they can all be interpreted on the same scale. If it doesn't, then it makes more sense to use a backbone and ignore the original weights, which would be misleading. So, it might depend a bit on what weights or "volume" mean in your use case.

Although I may not retain weights, I do think it's a great idea to retain other edge attributes. I'll put that on my to-do list for the next backbone update.

Sorry for the confusion about the code I suggested earlier. The version I posted will work if your input object is an adjacency matrix. Your solution is a nice one if you're starting with an igraph object instead.

[baslat]

Thanks, I just read it! I like the concept of ambiguity, I agree with your point that the weight on its on is insufficient to determine if an edge is strong or weak. However I'm not convinced that strong edges' weights should not be used subsequently. In this case my network is an origin-destination matrix of movements between suburbs across a city, where the weights are basically traffic volumes. This is a very dense network, so the backbone helps cut through the noise and see the strong edges. However, once the strong edges are identified by the backbone, the weights are still important as they can contribute to subsequent discussions and decisions, especially around prioritisation. A very simple, naive, example might be "which strong edge from this node should prioritised for infrastructure upgrades? The one which is most used (ie has the highest weight)". Basically I think edge weights have uses other than for determining whether an edge is strong or weak.

Either way, I appreciate your work on the package and the discussion, and hope you'll consider retaining edge weights (possibly as a function argument?).

[zpneal]

Thanks for checking out the paper. It's reassuring that the idea of ambiguity made sense. It's still in draft form, so I'm still on the fence about exactly how strong the recommendation should be about not using post-backbone weights (especially after seeing how you're using them).

Thanks also for sharing more details about your application, which is really helpful. I can certainly see how using the weights of retained edges might be helpful here. It seems like your application could be viewed as a specific case of a more general two-stage pruning approach: first apply backbone, then apply a local threshold to the remaining edges. In your case, the 'local threshold' involves looking at each node's strongest edge. Because your threshold is local, as opposed to global, it's very much still in the spirit of backbone models, and seems quite reasonable.

It also make me wonder, what would happen if we apply disparity filter iteratively: apply it, then apply it again on a resulting backbone that contains the weights of retained edges. We'd need to keep the weights to find out.

So, you've convinced me there are some potential uses for post-backbone weights. When I get to updating the package so it'll retain edge attributes, I'll be sure to include weights on that list. I'll keep this issue open until I get to it, which might take a while. Thanks for this really helpful exchange!