xyjprc
commented
4 years ago Thank you @seanlaw for your interest, that is a great question.
HON does share quite a few similarities with RNN or other neural networks, in the sense that
- They can both capture higher-order dependencies in sequential data
- They both leverage some additional network structure to represent such information
However, to my understanding, they differ in the following ways:
- Design principle. HON was built the goal of being compatible with the existing network analysis framework, such that methods from PageRank, clustering, anomaly detection that designed for networks (graphs) can still run on HON and leverage the higher-order structures. RNN on the other hand, is designed to solve classification, aiming for higher prediction accuracies. While HON changes the behavior of random walkers to "smarter" random walkers, HON is not meant to serve as a "next step predictor".
- Network wiring. HON's edges are learned from the sequential data and can be sparse, or preserve typical real-world network structures such as hub-spoke. RNN's structures are usually pre-defined, grid-like, having clearly defined layers, and usually dense.
- Edge weights. HON learns all its edge weights from the trajectory data, being the observed transitional probability from one higher-order node to another. RNN's edge weights not necessarily the transition probability, but are optimized for the classification/prediction task.
- Nodes. HON's nodes map directly to the entities and has physical meanings. RNN's nodes may or may not have physical meanings.
- Visualization/interpretation. HON can be visualized similar to networks, with each node representing the entity of interest (ports in global shipping, people in social network, etc). My banner picture on LinkedIn is an example of visualization of HON. On the other hand, the detailed structure of RNN may not be equally interpretable.
- Target audience: HON originates from the network science community, and RNN originates from the machine learning community. Although there are recent work that bridge the two, for example, this recent paper When to Remember Where You Came from: Node Representation Learning in Higher-order Networks attempts to learn network structure from HON and use it as features for machine learning tasks.
Hope this comparison can help add some color to your question.
seanlaw
I was reading your Sci. Adv. paper and I was wondering how HON compares to recurrent neural networks. I understand that HON allows for the use of other network analysis methods such as clustering and ranking but, unlike the classical Markov property (first order dependency), modern RNNs now allow for longer term "memory" even beyond 5th order. Can you please comment on the pros and cons as you see it?