Week 2. Mar. 29: Deep Architectures, Training & Taming - Possibilities

[JunsolKim]

Pose a question about the one of the following possible readings:

“Dropout: A Simple Way to Prevent Neural Networks from Overfitting”. 2014. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov. Journal of Machine Learning Research 15: 1929-1958.

“Understanding the role of individual units in a deep neural network”. 2020. D. Baua, J. Y. Zhua, H. Strobeltc. PNAS 117(48):30071.

“Graph Structure of Neural Networks”. 2020. J. You, J. Leskovec, K. He, S. Xie. ICLR 119:10881-10891.

“The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks”. 2019. J. Frankle & M. Carbin. ICLR.

[maddiehealy]

Q: Do we have the ability to change where a network looks when making a decision?

The authors of Understanding the role of individual units in a deep neural network explain that they are concerned with what it is a network is looking for and why (notably being the researchers to focus on the “why” portion to this question). Since we have the capability to use models, such as salience maps, that determine where a network looks when it makes a decision. However, my understanding is that salience maps output a representation of where the decision was pulled from, but are not interactive and therefore do not allow the user to change where the source is.

To rephrase my question, I know we can find where the neural network looks when making a decision, but I am concerned with if users can then go in and change where the neural network is looking? I see benefits to allowing users to change where a network looks when making a decision, even if these models are not always the ones being primarily published/analyzed. For example, I could see how offering two models of the same data, where each model sources data slightly differently, would offer interesting discussions for fleshing out social/cultural data and strengthening eventual interpretations. I am interested to see what differences would yield in this experiment. Additionally, I think it would promote a more reflexive approach when creating neural networks where users gain more control, with the exchange of users now needing a higher understanding of the complex innerworkings of these models.

[Pei0504]

Background: The "Lottery Ticket Hypothesis" posits that within a dense, randomly initialized feed-forward network, there exist smaller subnetworks ("winning tickets") that, when trained in isolation, can achieve comparable accuracy to the original network within a similar number of training iterations. This hypothesis challenges the conventional wisdom that dense networks are inherently more capable of learning effectively. The hypothesis was tested through experiments involving the training of networks, pruning to identify these "winning tickets", and evaluating their performance in isolation.

Question: Given the findings from the "Lottery Ticket Hypothesis" research, which suggests that smaller subnetworks within larger neural networks can reach similar levels of accuracy as their larger counterparts, how might this influence future approaches to neural network design, particularly in the context of resource-constrained environments? Moreover, what are the potential implications for our understanding of neural network optimization and the role of overparameterization in achieving effective learning outcomes?

[kceeyang]

When comparing one-shot and iterative pruning approaches in “Lottery Ticket Hypothesis,” the authors pointed out that the "winning ticket" initializations with iterative pruning are computationally expensive. The authors suggested that one-shot pruning may help to identify without repeated training but also indicated the problems of speed and test accuracy of one-shot pruning at smaller networks. Is there any other potential solution that can solve the computational cost of identifying smaller winning tickets?

[Audacity88]

“Dropout: A Simple Way to Prevent Neural Networks from Overfitting”

I found the "Motivation" section, where the authors compare their dropout strategy with sexual reproduction and political conspiracies, fascinating. I wonder if there are any other examples of strategies from other fields, e.g. evolutionary biology or game theory, inspiring deep learning models or methodologies?

[guanhongliu2000]

"[Dropout: A Simple Way to Prevent Neural Networks from Overfitting"

Comparative Study of Dropout and Contemporary Regularization Methods: In the landscape of regularization techniques, including L1/L2 regularization, batch normalization, and early stopping, where does dropout stand in terms of efficacy, computational efficiency, and ease of implementation? Moreover, under what conditions or in what scenarios might dropout outperform these other methods, and are there synergistic effects when combining dropout with other regularization techniques?

[Marugannwg]

Understanding the role of individual units in a deep neural network

It looks like the work diving into interpreting the meaning of different layers of the model is fruitful, and I find it intuitively understandable, especially with the visual example presented, how the neural network structure progressive processes the element (like the shape, items, textual at a different location) of an image. However, I wonder if it is because we can visually perceive the image that makes the meaning of the cells somewhat interpretable. Is there a systematic way to evaluate the neurons on tasks other than computer vision?

[anzhichen1999]

For the third paper, how does the exploration of the graph structure of neural networks, including the identification of "sweet spots" and the relationship between a network's architecture and its performance, complement the insights provided in the textbook regarding the diversity and innovation in neural network designs?

[XueweiLi1027]

[Understanding the role of individual units in a deep neural network] I am impressed by how the authors breaking down the mechanisms of a neural network by exploring the semantics of individual hidden units. Their findings show that it seems like a deep neural network is not a complete 'black-box' after all. I am wondering whether approaches following similar a logic can be applied to understand other advanced models, like the transformer?

[erikaz1]

Understanding the Role of Individual Units in a Deep Neural Network (Bau et al. 2020) explores how some individual units of a network store concepts and representations that are "human-interpretable" and salient to the network's ability to classify information in computer vision. In the section detailing GAN performance, GANs understand which objects and configurations are physically plausible within a scene (e.g. not rendering a door floating in the sky). How does the model learn these constraints on object interactions and basic physical principles?

[Xtzj2333]

Understanding the role of individual units in a deep neural network I find the method of removing units to investigate their impact fascinating. I wonder if a similar approach could be used to interpret deep neural networks that do not process images? For example, can this method interpret deep neural networks about text?

[uc-diamon]

When implementing dropout are we masking tokens or whole sequences when dealing with text data?

[risakogit]

In "Understanding the role of individual units in a deep neural network", they measure the impact of removing each unit on the network’s ability of classifying each individual scene class. Is this common practice? Is this method also used in other tasks, such as speech recognition or text analysis?

[beilrz]

Lottery Ticket Hypothesis: the paper suggest that the performance of many pervious state-of-the-art can be achieved by a smaller network, and the authors speculate it is because of the these parameter have appropriate initial weight to enable them to be trained better. Then, I was wondering does this suggest that we should train a very large neural network at the beginning, and then pruning it into a smaller net, as it create more opportunities for the weights to win the lottery tickets.

[HongzhangXie]

The method from "Dropout: A Simple Way to Prevent Neural Networks from Overfitting" left a strong impression on me—I found it hard to believe that randomly "dropping" units from a neural network could enhance the network's predictive accuracy. I'm curious about the source of this improvement.

After reading the paper, my understanding of dropout is that each time dropout is used, it essentially extracts a "subnetwork" from the original, complete network for training, with different training iterations generating different subnetworks. This approach is akin to training multiple distinct models and averaging their predictions to enhance the overall model's predictive capability.

The authors mention that "Bayesian Neural Networks are the proper way of doing model averaging over the space of neural network structures and parameters." With sufficient computational resources, Bayesian Neural Networks perform better than dropout. However, the computational demand of Bayesian Neural Networks becomes unacceptably high as the number of variables increases. Dropout significantly reduces computational requirements at the cost of a slight decrease in accuracy. So, can we achieve similar effects through means other than dropout units? For instance, by introducing some randomness during each training session, or by slightly altering the neural network before averaging. Can we demonstrate that given a certain level of computational power, dropout performs better in most scenarios?

[HamsterradYC]

“Understanding the role of individual units in a deep neural network”. 2020. D. Baua, J. Y. Zhua, H. Strobeltc. PNAS 117(48):30071.

In this work network dissection provides a powerful framework for understanding the inner workings of deep neural networks, especially CNNsand GANs, by revealing the associations between individual network units and human-interpretable concepts. However, this method primarily focuses on the analysis of individual units. I am interested in its applicability and extendibility in broader application scenarios. Specifically, I wonder if this approach could help us better understand the internal mechanisms of GANs and, based on this in-depth understanding, whether it is possible to devise new strategies for defending against adversarial samples, thereby enhancing the model's robustness in the face of unknown attacks.

[CYL24]

Understanding the role of individual units in a deep neural network

The article shed light on how individual units within deep neural networks contribute to the interpretability and explainability of the decisions made by the networks, particularly in the tasks like object detection and scene classification, which could be argued as being super helpful in dissolving the concerns around deep learning being the "black box." Do you think, in real-life world, this understanding will enhance the reliability and transparency of AI systems hence a even higher acceptance rate in terms of applying AI systems?

[MarkValadez]

The paper on "Graph Structure" analyzes NN as graph representations, and through graph metrics to such as clustering to understand the structural effect of NN in performance. In the paper they observe a sweet-spot between clustering coefficient and performance. Since clustering coefficient is a measurement based on the presence of triads, I wonder if this count be taken one step further to understand the distributions of triad types as they relate to performance. This could provide insight on the design choice given that mutual and asymmetric links in triads would link latter layers to earlier layers. Does this seem reasonable? What might be some flaws or impossibilities to this kind of design aproach?

[hantaoxiao]

For the research talking about dropout,

1. How does the application of dropout techniques in neural networks impact the fairness of predictive models, especially in contexts sensitive to social biases like credit scoring or criminal justice? This question ties into the broader discourse on AI ethics and fairness, focusing on whether dropout helps or hinders the equitable performance of predictive models across diverse demographic groups.
2. How can dropout be effectively utilized in neural networks that model social policies or simulate social systems? Considering that social systems are inherently noisy and complex, could dropout introduce the necessary robustness in these models, allowing them to generalize better from simulation environments to real-world applications?

[Yu-TingWeng]

"Understanding the role of individual units in a deep neural network": Are there any limitations or challenges associated with the practical implementation of network dissection for model visualization?

[kangyic]

'Graph Structure of Neural Networks'

Performance of neural networks depends on their architecture is indeed new and interesting to me. I wonder despite average path length and clustering coefficient, are there other measures of graph that can give us insights on? Also, even after reading the paper, I understand the neural network’s predictive performance is approximately a smooth function of the clus- tering coefficient and average path length, but are there any explainations why is it working? Are there any solid reasons behind the phenomenon?

[La5zY]

Considering the role of individual units in deep neural networks as discussed, how can we apply the network dissection technique to improve the interpretability and transparency of models in domains outside of image classification and generation, such as natural language processing or audio signal processing?

[00ikaros]

What are the main benefits and drawbacks of using dropout as a technique to improve neural network performance, particularly in terms of addressing overfitting? Additionally, how does dropout manage to enhance generalization across different application domains while simultaneously increasing training time, and what are potential future directions for improving its efficiency?

[Carolineyx]

The Lottery Ticket Hypothesis presents a compelling idea that small, trainable subnetworks exist within larger, randomly-initialized networks. Given the complexity and variability of neural network architectures, what are the main challenges in consistently identifying these "winning tickets" across different types of networks and datasets? Furthermore, how can the process of pruning and identifying these subnetworks be optimized to minimize computational overhead while ensuring robust performance improvements in both training efficiency and final accuracy?

[icarlous]

The Lottery Ticket Hypothesis suggests that state-of-the-art performance can be achieved by smaller networks because certain parameters have suitable initial weights that facilitate better training. Does this imply that we should initially train a very large neural network and then prune it to a smaller network, as this increases the chances of the weights “winning” the lottery tickets?

[Brian-W00]

What are the potential challenges and opportunities of applying Dropout technology in real-time learning systems? These systems have faster requirements for model updates and adjustments. How will Dropout technology perform in this environment?