Week 6. Apr. 26: Auto-encoders, Network & Table Learning - Orienting

[JunsolKim]

Post your questions here about: “Network Learning” & “Knowledge and Table Learning”, Thinking with Deep Learning, chapters 11 & 12

[maddiehealy]

In Chapter 12's HinSAGE example, ReLU and softmax are the chosen activation functions. Are there any combinations of activation functions to be wary about in this context?

[Marugannwg]

• Regarding analyzing networks through graphs, I think one essential question (before applying techniques) is -- how should we define each node and edge? (Any interesting examples beyond considering node as person and edge as frequency/relatedness?)

• Regarding heterogeneous graphs, does that mean we can have different types of nodes and types of edges on the same graph and process it with one algorithm? (e.g. If I have a graph with nodes representing people and multiple different types of edges (frequency of conversation, number of collaboration on project, student/teacher/job-title relationship...), what might I get out of it when I process it with HinSAGE?

[kceeyang]

The problem of the GNN transductive learning method was mentioned while discussing Deepwalk in Chapter 12 — that is, when a new node is added or a node is removed, the model needs to be retrained from scratch. As opposed to this, the GNN inductive learning approach, like GraphSAGE, would be able to test the trained model on new, unseen nodes. Although both methods have certain limitations, would this retraining effort and high memory and computational cost make the GNN inductive learning approach generally better than the GNN transductive learning method?

[XueweiLi1027]

Network Learning Unlike running models on tabular or text data, network data often has clustering. Some clusterings might have more influence on the predicted outcome than others. During the network learning process, is it possible to manually assign weights to different clusterings?

[Pei0504]

Given the detailed exploration of Graph Neural Networks (GNNs) and their application in the document, how does the performance of Graph Attention Networks (GATs) compare with that of traditional Graph Convolutional Networks (GCNs) when applied to dynamic graphs, particularly in scenarios involving frequently updated network data such as social media interactions or traffic networks?

[guanhongliu2000]

How do hyperbolic spaces, as a specific type of non-Euclidean geometry, provide advantages over traditional Euclidean embeddings when modeling hierarchical data structures in networks? Could you discuss the intrinsic properties of hyperbolic spaces that make them particularly suited for capturing the complexities of hierarchical relationships, such as in social networks or biological systems? Additionally, what are the computational trade-offs and challenges associated with implementing hyperbolic embeddings, particularly in terms of scalability, computational complexity, and integration with existing machine learning frameworks? How do these factors affect the practical deployment of hyperbolic embeddings in large-scale applications?

[risakogit]

What are some examples of tasks that bridge the gap between network clustering and classification?

[CYL24]

In chapter 12, it seems that GraphSAGE outperforms shallow node embeddings like DeepWalk and Node2Vec by considering both graph structure and node attributes, which leads to more informative node embeddings that capture both local and global patterns in the graph. In that case, how sensitive is GraphSAGE's performance to hyperparameters such as layer sizes, sampling techniques, and optimization algorithms? Are there best practices for tuning these hyperparameters to achieve optimal performance?

In addition, one crucial step in GraphSAGE is sampling positive and negative node pairs based on their co-occurrence within a context window in the graph. How does the size of the context window impact the sampling procedure and the performance of GraphSAGE? Are there any trade-offs between a smaller or larger context window size?

[Xtzj2333]

Last week's papers on social simulacra also investigated social networks in communities such as Reddit. I wonder how social simulacra could be complement network learning in order to predict and interpret social network phenomena?

[uc-diamon]

How to apply graph modeling to other datasets that do not come in the form of nodes and edges? Moreover, how do we determine what is a node and what is an edge and their connections?

[HongzhangXie]

In this chapter, I learned how to use methods like DeepWalk to detect communities and plot graphs between them. I am curious about how to evaluate metrics for graph neural networks. What are some of the key metrics used to evaluate the performance of graph neural networks, and how do they differ when assessing tasks such as node classification versus community detection?

[anzhichen1999]

How can inductive graph embedding models, particularly GraphSAGE, be optimized to maintain high accuracy and efficiency in evolving social networks without losing the characteristcs mentioned in the DeepWalk paper?

[Brian-W00]

This chapter mentions various types of graph neural networks (GNN), including graph convolutional networks (GCN), graph attention networks (GAT), and other unsupervised deep graph learning methods. In practical applications, which tasks do these networks usually perform best?

[mingxuan-he]

Recently there are some applications that use knowledge graphs as the knowledge base for LLM agents and they got better retrieval accuracy compared to using simply chunked documents. But currently most knowledge documents exist in the unstructured format. Is it possible to convert unstructured data into knowledge graphs? If so what tools/algorithms would best at this task?

[erikaz1]

In class, we briefly touched upon the use of networks for causal analysis. Do networks need to achieve a particular structure/size prior to applying causal assumptions?

[hantaoxiao]

GNNs are bridging the gap between theoretical computational models and practical, real-world applications by enabling more accurate modeling of relationships and interactions within networks. This capability could significantly improve our understanding of complex systems, from biological networks to social dynamics, potentially leading to breakthroughs in medicine and social science. How do advancements in graph neural networks (GNNs) impact the ability to model complex real-world networks like social interactions or protein structures?

[00ikaros]

What are the key methods for modeling, predicting, and generating network or graph data, and how can various types of data be represented as networks? How do common network tasks like link prediction and community detection benefit from embedding nodes, attributes, and edges into dense representations? Additionally, what are the capabilities and applications of contemporary graph neural networks (GNNs) such as Graph RNNs, Graph CNNs (GCNs), and Graph Attention Networks (GATs), and how do non-Euclidean geometries enhance the representation of hierarchies within network models?

[HamsterradYC]

In the context of using non-Euclidean geometries for hierarchical data representation, how do these approaches compare with traditional methods in terms of computational efficiency and accuracy? Are there specific types of network data or applications where these methods particularly excel?

[Carolineyx]

I'm curious about the impact of different graph embedding techniques on the performance of graph neural networks. How do methods like DeepWalk, Node2Vec, and GraphSAGE compare in terms of their ability to preserve the structural and semantic properties of the original network data? What are the key considerations for choosing one embedding method over another for specific types of graph-related tasks?

[beilrz]

One question I have about graph neural net in general is how do we design the GNN in the first place, assuming the connections (edges) are not given. For example, if we want to model a causal mechanism through GNN, what should we include as edges, if we only have the information about the nodes? Is there some ways to construct the network automatically through learning?

[icarlous]

What are the primary techniques for modeling, predicting, and generating graph data? How can diverse data types be networked? How do embeddings improve link prediction and community detection? What are the functions and applications of Graph RNNs, GCNs, and GATs, and how do non-Euclidean geometries enhance hierarchical representation in networks?

[MarkValadez]

In the case of Auto Encoders, what is the difference of using an explicit Embeddings layer vs using the output transformation of something such as a Dense layer? Does it change the accuracy or theory behind the interpretability?