Week 4: Apr. 12: Text Learning - Possibilities

[JunsolKim]

Pose a question about one of the following articles:

“The Geometry of Culture: Analyzing the Meanings of Class through Word Embeddings.” 2019. A. Kozlowski, M. Taddy, J. Evans. American Sociological Review.

“Aligning Multidimensional Worldviews and Discovering Ideological Differences”. 2021. J. Milbauer, A. Mathew, J. Evans. EMNLP.

“Who Sees the Future? A Deep Learning Language Model Demonstrates the Vision Advantage of Being Small” 2020. P Vicinanza, A. Goldberg, S. Srivastava.

“Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases” W. Guo, A. Caliskan. arXiv: 2006.03955.

“Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases.” 2020. “Can Pandemics Transform Scientific Novelty? Evidence from COVID-19.” 2020. “Application of Deep Learning Approaches for Sentiment Analysis.” 2019. “A deep learning model for detecting mental illness from user content on social media” (2020) “Using Word Embeddings to Analyze how Universities Conceptualize "Diversity" in their Online Institutional Presence” (2019) “A mathematical theory of semantic development in deep neural networks” (2019)

[XueweiLi1027]

Using data from subreddits, “A deep learning model for detecting mental illness from user content on social media” (2020) develops six independent binary classification CNN models to predict whether a user demonstrates signs of mental illness, including depression, anxiety, bipolar disorder, BPD, schizophrenia, and autism. They collected both training data and testing data from the six subreddit sections. Their models overall reach accuracy rates ranging from 75.13% (depression) to 96.96%(autism).

Though impressed by their application of CNN models in detecting mental health issue, I am wondering why does the authors only apply the models to data coming from the mental-illness related subreddits? I feel like their claim would be more convincing if they apply the models trained by data from the above subreddits to other reddit sections or even other user generated content social media platforms, like Facebook where users tend to post long texts.

[uc-diamon]

Regarding "A deep learning model for detecting mental illness from user content on social media", this is a case where our labels might not all be accurate, so how do we account for this? Also, how do account for comorbidities?

[maddiehealy]

I read Using Word Embeddings to Analyze how Universities Conceptualize “Diversity” in their Online Institutional Presence. They employ Word2Vec in this study and it made me wonder about other potential implementations/customizations of Word2Vec that I am unaware of.

If the words “home”, “residence”, and “dwelling” were placed in closed proximity to each other on the vector space, we can (via human interpretation) quickly discern that they are grouped because they are all synonyms for "house" and Word2Vec picked this up by analyzing context. Then, moving forward with our analysis, we know that this portion of the vector indicates synonyms for house. However, I wonder if Word2Vec can determine this label by itself? Does Word2Vec have the ability to assign these grouping labels, or is human intervention for labeling synonym groups the only current path?

[mingxuan-he]

The Intersectional Biases paper raises an important point that language embedding models (and therefore language generation models) inherit human biases and stereotypes in an intersectional setting. I'm wondering if modern LLM tuning methods with human feedback (e.g. RLHF) would worsen these biases or reduce them.

[kceeyang]

In “A mathematical theory of semantic development in deep neural networks”, the authors were able to use a deep linear network model to capture the learning dynamics of diversity of phenomena involving semantic cognition and revealed mathematical definitions of several aspects of semantic cognition based on the model result. Although many nonlinear features do not require neuronal nonlinearities, it seems like some semantic phenomena that require complex nonlinear processing were still not able to be explained due to the linearity of the model. I’m curious: is there maybe a dual-process model that comprises both linear thinking and nonlinear thinking to capture all phenomena?

[Pei0504]

The study "Who Sees the Future? A Deep Learning Language Model Demonstrates the Vision Advantage of Being Small"underscores a reevaluation of the traditional belief that larger firms with more resources are the main sources of innovation. The findings suggest that agility and flexibility associated with smaller firms play a crucial role in pioneering novel ideas that later become mainstream. How generalizable are the findings across different industries? The study focuses on publicly traded firms; how might these insights apply to startups or industries with rapid innovation cycles like technology or biotech? The use of BERT to analyze conversational text is a methodological innovation that offers a new way to measure visionary capacity beyond traditional metrics like patents or product launches. This could broaden the analytical tools available for understanding innovation dynamics. How does the context in which a firm operates influence its ability to be visionary? For example, how do regulatory environments or economic conditions affect a firm's capacity to translate visionary ideas into practice?

[guanhongliu2000]

In “Aligning Multidimensional Worldviews and Discovering Ideological Differences”, what are the possible limitations might these models have in representing a community's ideological spectrum?

[risakogit]

"A Deep-Learning Model Of Prescient Ideas Demonstrates That They Emerge From The Periphery"

How do the principles of contextual novelty and foreshadowing future domain evolution be applied in other fields, such as technology, where novel reinterpretations of existing frameworks could happen?

[Xtzj2333]

“The Geometry of Culture: Analyzing the Meanings of Class through Word Embeddings.”

This paper examined how concepts shift in meaning with time. I wonder if a similar method could be used to examine how concepts shift in meaning across cultures? For example, do Chinese Americans use the word 'education' in the same way as European Americans do?

[anzhichen1999]

How does the use of BERT for detecting prescient ideas compare in accuracy and computational efficiency to other deep learning models like LSTM and Transformer models without bidirectional capabilities?

[HamsterradYC]

“Application of Deep Learning Approaches for Sentiment Analysis.” 2019. Although using specific vocabulary and lexicon can effectively extract potentially relevant data from social media posts, this method may not accurately capture the true psychological state of the posts. Despite the mention of relevant psychological and emotional vocabulary in the literature, these words do not always accurately reflect the corresponding psychological states. In a previous project, I used an advanced language model like GPT, which, with its self-learning and correction capabilities, improved the accuracy of identifying and assessing mental health states in posts and corrected incorrect labels. However, I found that as the volume of data increased, the model's responses became less precise, and the errors seemed to expand. What could be the reason for this phenomenon?

[CYL24]

A deep learning model for detecting mental illness from user content on social media In the article, the authors mentioned several limitations in their study, such as not considering socio-demographic factors, regional differences, and additional mental health conditions. What are some methods/approaches that can be used to improve the model performance (which factor one may anticipate as important?)

[Marugannwg]

I'm reviewing "Aligning Multidimensional Worldviews and Discovering Ideological Differences" (2021); I found the concept of comparing multiple (similar) communities very interesting -- as a video game player, the comparison between League of Legends and Dota2 community is very vivid.

I'm more curious about how to handle particular words with different esoteric meanings in the communities. e.g. "camping" can have drastically different semantic and sentimental differences in different game communities (a technical or an insulting word). How can the unique meanings of these words be traced under different contexts (given that these meanings are newly invented and not contained in the resource of language model?)

[HongzhangXie]

The paper titled "Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases" introduces a fascinating method for identifying biases through corpora. I am interested in how we can distinguish between biases and facts in a corpus. For example, the statement "men are taller than women" might be considered biased, while "according to statistics, the average height of men is taller than that of women" appears more like a factual statement. How can we better differentiate between text containing negative emotional biases and neutral factual descriptions?

[hantaoxiao]

The Geometry of Culture: Analyzing the Meanings of Class through Word Embeddings. How can word embeddings be utilized to analyze cultural dimensions such as social class? What are the main advantages of using neural network-based word embedding models over traditional methods of text analysis in cultural studies?

[kangyic]

A DEEP-LEARNING MODEL OF PRESCIENT IDEAS DEMONSTRATES THAT THEY EMERGE FROM THE PERIPHERY

I get the idea of perplexity score, but could it exits a prescient idea whose not composed of 'surprising' words or sentences? For example, I say, we love cats. But the context is a fairy tale and 'we' actually referred to cats hunters

[beilrz]

The “The Geometry of Culture: Analyzing the Meanings of Class through Word Embeddings.” paper suggests that wordnet anatomy pair dimension often result in meaningless result. Why would this be the case? also, assuming we have very limited knowledge about the field, how should we construct our dimension for analysis?

[erikaz1]

The central contribution of Aligning Multidimensional Worldviews and Discovering Ideological Differences by Milbauer et al. (2021) is the notion that two groups, despite talking about the same topics, seem to demonstrate "worldview misalignment". The notion of "misalignment" is operationalized as the distance between an original key word as it is defined and contextualized within one groups' lexicon to the definition and contextual usage within another group (page 4838 explains this wonderfully). While the results make sense to me, I wonder if there is more at play here than the clear "misalignment". I wonder if these other factors are not captured by the unit of observation "misalignment". This is because if we look at r/politics and r/the_donald, the alignment algorithm translates democrat in r/politics to republican in r/the_donald, but this doesn't tell us how each group actually views one or the other party. Section 4.5 notes that "communities for which we hypothesize a strong ideological disagreement...there is a strong similarity in topic distribution" (4836). However, if "democrat" is projecting to "republican", how is this indicative of "similarity in topic distribution"?

[00ikaros]

What are the primary advantages of word embedding models for cultural analysis, particularly in handling large corpora of digitized text, and how do these models compare to traditional methods like topic modeling and semantic network analysis? Additionally, how do word embeddings validate relational approaches to cultural theorizing and enhance our understanding of the complex and multidimensional nature of culture, enabling new perspectives on social dynamics and interactions?

[Carolineyx]

Vicinanza et al. propose that prescient ideas often emerge from the periphery rather than the core of a field. Given this counterintuitive finding, what are the implications for organizations or institutions aiming to foster innovation and identify transformative ideas? Specifically, how can they effectively balance attention between established core actors and peripheral innovators to maximize their potential for groundbreaking advancements? Additionally, what methodological refinements could enhance the detection of prescient ideas, particularly in rapidly evolving or highly interdisciplinary fields?

[La5zY]

How does the application of BERT to conversational text data from quarterly earnings calls enhance our understanding of the contextual novelty and prescience of visionary ideas in firms, and what implications does this have for the comparative advantage of small versus large firms in terms of innovation and market performance?

[icarlous]

In “Application of Deep Learning Approaches for Sentiment Analysis” (2019), specific vocabulary can extract relevant social media data, but may not capture true psychological states. I used GPT to improve accuracy in assessing mental health from posts, but accuracy declined with larger data volumes. Why does this happen?