7. Deep Learning to Perform Causal Inference - [E3] Saha, Koustuv, Eshwar Chandrasekharan, Munmun De Choudhury.

[lkcao]

Post questions here for this week's exemplary readings:

3. Saha, Koustuv, Eshwar Chandrasekharan, Munmun De Choudhury. 2019. “Prevalence and Psychological Effects of Hateful Speech in Online College Communities” WebSci: Proceedings of the 10th ACM Conference on Web Science, p. 255–264

[Dededon]

There are basically two different ways of doing: either to extract text variables and assess the treatment effect with simple OLS (Saha et al.'s paper), or to learn propensity scores with more complex machine learning architectures (CausalBERT). Saha et al.'s paper is treating text features as confounders, and use the keyword-wise pattern to do the matching. This paper's research design to measure hate measure the keyword probability compared with hate-speech community baselines is brilliant. My question is that what are the possible social games with large-scale texts that we can extract data of quasi-experiements from. Tracing reddit users is certain a good idea.

[Twilight233333]

I think the author's definition of exposure is as good as it can be. But there will still be some flaws, using the percentage of hate in comments that users participate in. Can we go a step further and try to see where these words appear? Would a distributed or centralized distribution make a difference?

[sborislo]

This was very interesting, and I think their approach to control for covariates was reasonable. Based on their findings, I wonder: Is it the initial exposure to hateful speech that causes the increase in stress, or is there a cumulative effect? It wasn't clear to me based on the graphs, which seem to suggest that there is not a cumulative effect. Figure 4 also seems to suggest that those more frequently exposed to hateful speech (people who play video games) exhibit high endurance. If there is no serious cumulative effect---and we believe some exposure to hateful speech in some area is inevitable---is "sheltering" to hateful speech overall not worth the effort?

[yuzhouw313]

I found this multi-stage research by Koustuv et al. to be comprehensive and illuminating. The study delves into hateful speech on college Reddit forums, categorizing distinct patterns of such speech, quantifying exposure to hateful comments, assessing stress levels, and ultimately analyzing the causal relationship between these factors, alongside an examination of psychological resilience. I was particularly intrigued by the College Hate Index (CHX) they introduced. Unlike conventional methods that rely on a simple count of keywords or the proportion of hateful terms, they calculated CHX using the ratio of normalized hate measures in college subreddits to those in banned subreddits. This approach raises questions about their rationale for setting the hate measure in banned subreddits as an upper benchmark. What advantages and disadvantages does this method offer as a proxy for gauging online hate sentiment?

[volt-1]

In analysis of the psychological impacts of hateful speech using a causal inference framework, the inherent issues of self-selection and echo chamber effects in online communities pose significant challenges. These factors can potentially distort the authenticity of causal relationships. How did you identify and adjust for these potential confounding variables to ensure the accuracy and reliability of the causal inference results?

[yunfeiavawang]

To compute the College Hate Index (CHX), the authors collected the comments of college communities and measured the prevalence of hate speech. I am not sure if the corpus selection is reasonable because the comments hate tendency highly relies on the originator. Controlling the linguistic features in the original posts seems like a necessary intervention before taking the CHX calculated from comments as the degree of hateful speech. Furthermore, they can also take a step forward, investigating if users with specific characters are more susceptible to other's hate emotions and easier to ignite. Nevertheless, the paper is an excellent example of applying supervised learning to detect causality between hate speech and mental health of college students.

[Marugannwg]

The matching-pair design using Reddit profile drew my attention. At first glance, it sounds (somehow) impossible for me to really sample those users how had not been exposed to hate speech -- how do you know what Reddit posts they read or engaged in? Returning to the paper, I realized that the definition of expose is more closer to commenting: the matching pairs were drawn among those who commented on hateful thread versus those who didn't, given extra population restrictions...

Also, I'm curious about the statistical significance of those psychological analyses. (I didn't find the p-value, effect size and detailed method report of how they obtained those results (e.g. the personality measurement))

[ethanjkoz]

I thought their method of labeling stress exposed to college students was very clever. The authors note they employed a supervised ML technique (linear SVM with 5-fold CV) to label posts as high or low stress (using r/stressed as examples of ground truth for stressed posts and posts on Reddit landing page for non stressed ground truth). One concern I had lies with their assumption that posts in r/stress necessarily deal with stress. Thinking about my own corpus on reddit posts from r/Adoption and r/Adopted, a small fraction of posts and many more comments have nothing at all to deal with adoption. I feel like a similar case might be happening here too. I would like to know more about how these posts were selected because we are not told whether these posts were chosen by experts as good examples or at random. I suppose most of my worry is somewhat abated by the 81% expert validation accuracy of the model but I am still not certain.

[naivetoad]

What specific online platforms outside of Reddit are significant sources of hate speech for college students, and how do these platform compare in terms of content and psychological impact? Broadening the scope to include other platforms could provide a more comprehensive view of the online landscape.

[yueqil2]

This research depends a lot on a high-precision lexicon from two research studies on hateful speech and social media and then adopt a pattern (keyword) matching approach. Considering my research, I wonder how to find high-precision lexicon for other topics, such as conspiracy theory or nationalism.

[QIXIN-LIN]

The College Hate Index (CHX) presents a fascinating approach, deriving its metrics by comparing the extent of normalized hate-related content within college-themed subreddits against that found in banned subreddits. In contemplating the development of an index ourselves, it's crucial to consider the mechanisms through which we can validate its effectiveness. Additionally, it's important to be aware of any critical factors or considerations that might impact its reliability and integrity. What measures can we implement to ensure our index accurately reflects the phenomena it's designed to measure?

[donatellafelice]

copied in the wrong question -- updated: I did not understand how the authors trained their log regression classifier for prediction. "we train a logistic regression classifier that predicts the propensity score (p) of each user using the above described covariates as features" how would the users use the covariates described and how would the model identify that?

[muhua-h]

How does the lexicon-based approach account for the context in which certain terms may be used non-hatefully, and how might this affect the accuracy of hateful speech identification?

[alejandrosarria0296]

The authors mentioned that an optimal approach to determine causality in this scenario would imply randomized control trials but that this is impossible due to the logistical and methodological limitations of the study. If these were to change to enable for RCTs, how would this method look when based around text as data?

[cty20010831]

This is a very practically meaningful paper that take advantages on computational methods of textual methods to measure the prevalence and psychological consequences of online hate speech among college communities. My question would be how was the lexicon they used for keyword matching originally devised? Were there computational methods involved (if so, what type of methods were used) or manually done?

[runlinw0525]

I wonder that in the study on College Hate Index (CHI) and its impact on stress expression in college subreddits, what were the key differences in language and personality traits between low and high endurance users when exposed to hateful speech, and how did these traits correlate with their psychological responses?

[chenyt16]

From the current perspective, the method of using lexicons to detect hate speech is not comprehensive enough because it can only filter out explicit hate speech. In recent years, algorithms for detecting implicit hate speech have also been developed. I want to know if the results of this article can be replicated if we include implicit hate speech and whether people's psychological states differ when exposed to explicit hate speech and implicit hate speech.

[HamsterradYC]

The study evaluates the topical similarity between treatment and control users to minimize confounding by interest. However, this approach might inadvertently exclude individuals who encounter hateful speech outside their primary topics of interest, potentially underestimating the impact of hate speech exposure on stress. Does the methodology adequately account for the nuanced ways hate speech exposure might affect individuals, given that its relevance and impact may extend beyond their typical topics of interest?

[Brian-W00]

After reading about how online hate speech affects students' stress levels, I wonder, how can universities support those who face hate speech online but are not part of any study. Are there effective strategies or programs already in place in some colleges?

[icarlous]

When considering developing a similar index, it's vital to explore validation mechanisms and acknowledge factors affecting reliability. What measures can be taken to ensure our index precisely mirrors the phenomena it aims to measure?

[Caojie2001]

For this research, I have a question that I have always been curious about when reading other ML research. The author mentioned that the performance of the keyword-matching approach is similar to that of the advanced ML classification methods. In my experience, this kind of expression is quite common in my ML-related research. Therefore, I wonder if there is a relationship between specific characters of the classification task and the selection of primary and advanced classification methods.

[beilrz]

This paper rely on a baseline to measure hate speech. I was wondering how can we construct such baselines for our own research task? For example, if we want to measure if somebody is leaning liberal or conservative, should we construct baseline for liberal or conservative and conservative community, and determine what baseline the person is closer to?

[floriatea]

Given that students often use multiple social media platforms, how do the dynamics and prevalence of hate speech compare across different platforms? What role do platform-specific policies and community norms play in shaping these dynamics, and how can platforms collaborate to address hate speech more effectively? What steps can effectively mitigate the prevalence and intensity of hateful speech within college communities without infringing on privacy rights?

[Vindmn1234]

Given the significant impact of hateful speech identified in the study, particularly its correlation with stress expression and psychological endurance, how do you envisage online platforms and college administrations collaboratively working to mitigate these effects? Are there specific intervention strategies or technological solutions you believe could effectively reduce the prevalence of hateful speech and its psychological toll on students in online college communities?