Assignments for session 5

[simonsimson]

1 Read and comment

Read Paper: Sanders, Abraham, Rachael White, Lauren Severson, Rufeng Ma, Richard McQueen, Haniel Campos Alcanatara Paulo, Yucheng Zhang, John S Erickson, und Kristin P Bennett. „Unmasking the Conversation on Masks: Natural Language Processing for Topical Sentiment Analysis of COVID-19 Twitter Discourse“. Preprint. Health Informatics, 1. September 2020. https://doi.org/10.1101/2020.08.28.20183863.

Answer the following questions in a summary of 150 words:

• In which ‘mode of inquiry’ (Marres and Moats 2015) is the research project and paper operating?
• What are the issues that are silenced / cannot be grasped by this mode of inquiry?
  

2 Set up pipeline and comment In the folder 'Pipeline/':

• Setup the Jupyter notebook. A guide on how do this is provided in Setup.txt.
• Preprocess and embed your data with the Language Model
• Pick some comments you found interesting in your prior analysis. Get similar comments.
• Answer the following questions in a summary of 150 words:
  • What can the model do well? When does it fail?
  • How can you use it in your project?
• Add the folders 'data/' and 'output/' to .gitignore, so they are excluded from the repository
• Commit+Push your Notebook with outputs to GitHub: create a new folder named '[your_name]_assignment4'.
  

3 Submit assignments on Github (reply to this issue) until 2 Dec 12h00 (noon) Commentaries to 1 and 2 + URL link to your notebook

[satorus]

Reading Assignment I think the research was done in precautious mode. They mostly focused on the content of the discussion while mostly disregarding the platform features at hand. They did not try to weed out any bot accounts or other platform-specific content, but on the other hand they did not try to analyze the platform specific dynamics at their core, disregarding retweets and not investigating how the platform dynamics play into the evolution of the discussion about masks. This also shows the first issue that is silenced by this mode of inquiry. The research analyzes the content of the discussion but pays no specific attention towards how the characteristic twitter dynamics can or actually do change the discussion. There is no look into how for example retweeting into different circles can influence the sentiment in a cluster or how likes and trending hashtags for some of the cluster keywords may influence the discussion by putting some tweets in the spotlight.

Pipeline Assignment The model does well in finding similar or neighbouring comments. It fails seemingly in printing these as a heat map, as in my results similar comments were found but the heatmap showed no similarity between comments. In my project, I can use the model to quickly find similar comments to comments that are relevant to my research topic or that spark discussion about this. It can also be helpful for finding similar tweets or other sorts I guess. It can also be a valuable tool to show how diverse the discussion in the comments/tweets/etc is, as many similar comments may indicate that the comment section is more of a “conformist” section, with many people just agreeing to the video contents. It may also help finding similarities across different video comment sections, matching discussions to the same topic together, which helps finding relevant data for my research project and showing relations between these comments and discussions.

Notebook

[Cloudz333]

Paper analysis:

• In which ‘mode of inquiry’ (Marres and Moats 2015) is the research project and paper operating?
• What are the issues that are silenced / cannot be grasped by this mode of inquiry?

In this study, I could not identify any activity correlated to bot detection. On the other hand, non meaningful data points that were not related with the debates were filtered out by selecting those tweets that contain at least one keyphrase in both categories: COVID-19 and mask-wearing. For this reason I assume that this research operates in Precautionary mode. Therefore I could not identify any form of affirmative and empirical approach. The fact that the researchers were not focussing on platform specific features or effects belonging to the media technologies, could in my opinion compromise the representativeness of the data. One aspect not taken into consideration by the approach used, is that the twitter API may introduce BIAS due to manipulation: “... it has been shown that samples of tweets obtained via the API reflect the general user content generation patterns of the complete Twittersphere accurately”. We know from the previous discussed research, that this may not be true.

Model analysis:

• What can the model do well? When does it fail?
• How can you use it in your project?

I could test the algorithm with some comments regarding different debates in the video and the results are surprising. The model is able to recognize similar comments quite well. For example, comments regarding the interviewee's abilities (“Who gave this man a degree”) or comments on the reliability of the television broadcast (“CBS is BS News”) are very well mapped in the vector space and produce very reliable results. Since my dataset is very small (around 70 comments) I could not find any comment that can be miss-clustered due to the discrepancy between the data used for pretraining and my data. The model obviously cannot recognize which idea is prevalent and also cannot gain a deeper understanding of the debate. That’s why I would rather use the model only to perform some explorative analysis in order to brainstorm which kind of debates are present in the dataset, and then also to find and filter relevant comments related to my research goal.

LINK TO THE JUPYTER NOTEBOOK

[yaozheng600]

Worked with @ChristyLau

Reading Assignment According to Marres and Moats, the mode of inquiry used in this paper is more like the precaution mode. The posts containing the given keywords were selected and then divided into 15 groups according to its semantic theme, and eventually split into 15 sub-groups which represented specific topics. Their inquiry method fulfils the goal of their work. But unlike what was described in the last paper, Sanders et al. filtered the posts that contain the keyword related to both COVID-19 and masks based on text-based keyword identification. From my point of view, their criterion is a bit strict. There could be some posts which only contain the „mask“ keyword but not the keyword of COVID because people take COVID-19 as the common sense which do not need to be pointed out. Besides, unlike the precaution mode used by Marres and Moats, the method used in this paper does not include the step of bot cleansing. So the representativeness of chosen posts is still in doubt.

Pipeline Assignment
This model performs very well when the tested comments are short. But if the comments are long, the model can't compare them so well. In my opinion, the heat map is not appropriate to show a large set of comments and using the comments itself as x/y-axis makes the chart complex. Besides, this model can recognize different languages, which makes the debate possible in a range of worldwide.
The main task of our project is to analyze the attitude of comments (positive, general, negative). First I thought that this model could be used to cluster similar comments. This makes subsequent comment analysis and statistics easier (we only analyze a few random comments in a category, instead of analyzing all comments). We will implement the feasibility of this idea in future projects.

Link to the notebook

[isaschm]

Reading assignment: Sanders, White, et al (2020) used Twitter to garner data on public opinions on wearing masks during the pandemic, an issue relevant to all tiers of society. While that would suggest they view social media as a relevant place of interaction, a few decisions in the way they approached the data reveal their mode of inquiry to be ‘precautionary’. They discarded elements of a common twitter controversy, such as retweets, the number of likes, and threads as not relevant. They also chose to not analyze hashtags but keyphrases. While that makes sense considering the number of tweets, it suggests the researchers were not looking for platform specific interactions. A question aiming at interrogating the role of Twitter in the controversy could not be answered this way. Discarding the retweet, while keeping the original tweet is telling of approach to social media and more specifically Twitter. Quoting a retweet is often done is a way that reveals the users’ attitude towards the issue at hand. The quoting is done in a mocking, agreeing or ambivalent manner.

Pipeline: The model proved to very good recognizing different languages, both in the way that it kept comments of different languages apart and by keeping short comments of several languages with a single message, such as “rip florida” together. I would also argue that is a relatively useful model in general. Even when randomly picking a comment from my dataset, the similar comments displayed made sense to me. The heatmap seems to me as not useful with large samples. The more data is being compared the less informative is the resulting map. It would be interesting if one were to be interested in a small and carefully selected sample of one’s data. I would also assume the model would not be useful in a setting were the data is relatively homogenous and the task would be to discern the tone of a comments. As the notebook is setup, I would use the model if I was interested in finding similar comments to a comment that I thought of myself and not so much to get insights into a general opinion

Link to notebook

[iraari]

1. Read and comment I suppose the paper by A. C. Sanders et al. adopts a precautionary approach. The method is well suited for deploying social media analysis for social research and that’s what the authors do: they focus on content by building clusters (and subclusters) representing discussion themes and specific topics within each theme. In doing so, they filter tweets by keywords (not hashtags though), clean tweets by removing URLs, and discard retweets. They do not take into account the number of likes or followers, to whom tweets were shown. In other words, in the article it is not considered how platform-specific features may influence digital issue analysis (which is one of the main characteristics of the affirmative approach). The chosen mode of inquiry has its flaws, e.g. it cannot investigate connections between content and users, relations between different users, or the difference between knowledge claims and the political/social positions of users.

2. Set up pipeline and comment The Transformers model is good at finding similar comments. It works better on shorter sentences, especially if the comments refer to the guest or host (it’s a podcast). The longer the initial comment, the more diverse the similar ones are. It’s not a flaw in the pipeline, but a research issue, since the problem of sentence similarity is not yet as well studied as the problem of the semantic similarity between two words or longer texts. However, I could use this method to cluster comments on topics. Although the heat map is not very practical/convenient for exploring a large number of comments, it is quite interesting to see it and get an idea of the dynamics of comments, e.g. how similar the nearby comments are. For example, I could select comments from one discussion and see how similar the first comments are to the last ones or find out if the top comments relate to the same topic etc.

3. URL link to the notebook Jupyter Notebook

[adrigru]

Reading assignment

According to Marres and Moats 2015, the authors of "Unmasking the Conversation on Masks: Natural Language Processing for Topical Sentiment Analysis of COVID-19 Twitter Discourse" paper operate in the precautionary mode. They do so by ignoring the platform-specific artefacts when analysing the data. They use the Twitter 1% API, which was proven prone to manipulation. Moreover, the authors miss the whole picture by selecting tweets that contain at least one key phrase in both categories. Therefore, they do not consider tweets related either just to wearing masks or only to covid. On the other hand, they do not examine if the tweets contain other hashtags related to marketing or other topics. As a result, the corpus might contain tweets unrelated to the subject, that highjacked the popularity of given hashtags, which might have an impact on the final results.

Furthermore, the authors disregard every platform-specific features. They exclude retweets from the data and ignore the likes. However, tweets from prominent users, that have many likes will have a significant impact on the course of the discussion and the sentiment compared to usual tweets.

Pipeline assignment

In general, the Universal Sentence Encoder performs well for natural language processing tasks such as language translation, semantic analysis, outlier detection or fact-checking. It works by encoding the linguistic structures in the form of vector embeddings which represent the dependencies between individual words and sentences mathematically. The resulting embeddings allow for numerous further analytics. In the given assignment, we computed a similarity matrix for YouTube comments using angular similarity distance. However, there is a lot more that one can do using the embeddings. One exemplary application is sentiment-analysis or a hate-speech detector. As good as the model is for some tasks, it also has its drawbacks. First of all, it can only be as good as the data is, meaning that noisy data will make it more difficult to obtain meaningful results. Second, it has no deeper understanding of language. Therefore, it won't understand irony or other literal techniques often used in YouTube comments.

Notebook

[Rahaf66]

Reading Assignment: In the light of the work of "Marres and Moats 2015" and since the paper "Unmasking the Conversation on Masks" is interested in deploying social media analysis for social research, it is operating in the precautionary inquiry. The data was collected using API, filtered just according to the content to be strongly related to COVID-19 and mask-wearing and cleaned by removing URLs and non-punctuation characters. In addition, there was no study of the impact of relations between users, number of likes, or bot detection. It shows that the project was done irrespective of analyzing platform-specific dynamics in order to detect issue activity (which can be explained by the affirmative mode). We can also talk about the biasness of API sampling we have read about last week and the researchers did not take it into consideration which increases the potentiality of skewness in the results.

Model analysis:

The model had a bad performance when I tried it with a large number of comments (about 22000), therefore I applied it for 1000 comments that present of course just a part of the debates. The model is very good at getting the most similar comments of a comment especially when the central comment is not long, otherwise, the results were less precise. The heatmap presents only one topic (related to water issues) although it seems reasonable. However, it did not work well for other samples.

Text embedding is an essential task in natural language processing, and the model can be used in many applications of NLP processing especially applications that depend on the concept of similarity such as topic modeling. So, in my project, I can make use of the model when the aim is clustering the comments according to their similarity, detecting the most similar comments to one comment as we did, topic modeling, or sentiment analysis.

URL link to the notebook Notebook

[adrianapintod]

Reading Assignment: The research in this paper seems to operate in a "Precautionary mode" as the initial data obtained from "Twitter's API 1%" is filtered by keywords to reduce the data on which the analysis was based. However, it did not carry out an exhaustive cleaning of the data, which is the core idea of this mode of inquiry. Authors appeared to omit the treatment of bots or any other platform artifact to deal with the issue of platform bias. Moreover, limiting the extraction of the final data to data filtering by keywords does not exclude advertisement Tweets which may influence the analysis. The analysis did not follow an "Affirmative mode" of inquiry as they did not consider the platform-specific dynamic by discarding retweets, that would reveal interesting insights for the analysis.

Pipeline and comment: Executing transformations on texts (sentences, words) to convert them into numerical vectors (Text embeddings) so that they can be easily analyzed is an important task in Natural Language Processing. However, the Universal Sentence encoder does quite well the job of finding semantic similarity among comments, disregarding the type of sentence encoding model (DAN and Transformer) chosen when trying with different comments. Although, in this case, the encoder performed well, I can imagine that sentences with considerable grammatically incorrect syntax, as usually seen on Youtube or other platforms interactions, as well as other literary figures such as sarcasm, satire, etc., will negatively influence the outcome of the analysis. This technique, already available and implemented in the TensorFlow-hub, would be useful in the final project, leveraging it for the same purpose of this assignment, that is, finding and clustering semantically similar comments.

Link to the notebook

[SabinaP17]

1. Reading assignment

I believe that the researchers applied in this scientific paper a precautionary approach mode to analyze data extracted from tweets about the topic of mask usage during the COVID-19 pandemic in the first five months of 2020. Their analysis focused mainly on the content of the extracted tweets and paid no attention to possible data manipulation and data bias due to bot activity, over-represented Twitter accounts or other platform related technical vulnerabilities (the latter aspects were discussed in the paper by Pfeffer, Mayer & Morstatter, 2018). The tweets were collected from the Twitter streaming API in a “semi-selective approach”: they were filtered two times, the first time directly after collection by keywords “loosely associated to COVID-19” (Sanders et al., 2020) and the second time using Elasticsearch by “the criteria that a tweet must include at least one keyword indicating it is strongly associated to COVID-19 and at least one keyword indicating it is strongly associated to mask-wearing” (Sanders et al., 2020). From my point of view, the filtered data doesn’t manage to paint the whole picture, so to say, because other important factors were completely discarded from the analysis. For example, the retweets of the collected tweets and their role in the overall discussion were not taken into consideration into the analysis; nor were the platform specific dynamics (for examples, hashtags or likes) when analyzing how the discussion about the topic of mask usage evolved within the five months or how certain sentiments might have been instigated or activated.

[raupy]

1. Reading Although I couldn't find any effort in bot detection, non meaningful data was filtered out by selecting those tweets that contain at least one keyphrase in both categories: COVID-19 and mask-wearing. That is why I assume that this research operates in Precautionary mode. By this mode of inquiry, there is no platform specific behaviour in consideration, e.g. the retweeting of tweets. Also it was ignored that Twitter's 1% API may be biased as we know from the previous paper. On the contrary, the authors claim that the 1% API provides a good and representative sample.

2. Pipeline Well, I think it depends on the definition of 'similarity'. For me, similar comments to a comment, that critizises the AfD and those people who say that climate change is a lie, would not be AfD supporting comments. But of course, they are talking on a similar topic, so one could say that they are similar. The comment was also rather long I have to admit and I think the model works best for short statements. It was pretty good in finding the Claudia Roth related comments. For my project I think it that the model could be helpful as a preprocessing step but I would have to manually look again at the comments that were found and sort them. Notebook

[mrtobie]

1 Read and comment

• In which ‘mode of inquiry’ (Marres and Moats 2015) is the research project and paper operating?
• What are the issues that are silenced / cannot be grasped by this mode of inquiry?

The given Paper has to be written in precautionary mode since there is no discussion about platform specific subjects. Platform specific topics are also the issues that were not tackled by the paper. The authors just discuss the mechanism of the Twitter API. Unlike an earlier paper (Pfeffer, Jürgen, Katja Mayer, und Fred Morstatter. „Tampering with Twitter’s Sample API“. EPJ Data Science 7, Nr. 1 (Dezember 2018): 1–21.) the 1% sample of the Twitter-Data is presented as accurate when it comes to represent the worldwide Twitter-Data. While the methodological process of the analysis of the content is very cleary discribed, the platform-specific issues are not mentioned in a word. When dealing with social-media data it is crucial to keep in mind that there is an effect on the content of the data as well as the sampled data itself.

2 Set up pipeline and comment

• What can the model do well? When does it fail?
• How can you use it in your project?

The modell does really well when it comes to compare comments, that are written in a fairly simple structure. Also short sentences are handled better than long ones. If a sentence is short, the modell compares it as it should by content of the message. If the sentence gets too long, it basically finds other comments, that are also long but does only pay little attention to the content. In the upcoming project this modell is good to get a feeling for how widely spread the opinions of the commenters are. Since our project is about single individuals it could be used to track other users that have similar opinions as the observed one. Other than that this modell might not be the perfect solution to use in our project.

Jupyter Notebook

[Alioio]

Reading Assignment:

The authors of this paper clearly focus on the content of the data while discarding platform specific attributes like retweets and are not making use of features such as likes. This brings me to the conclusion that they do operate with the precautionary approach.

To my understanding liking a tweed expresses confirmation and agreement with the tweet. Therefore using another approach likes could had been used for the trend analysis part of this research (complementary to the approach which is already used). Another issue which had not been investigated is detecting possible data bias. The data which is used in this work derives from twitters Sample API which is prone to data manipulation.

With further pre-analysis of the data, such biases for example from bots could had been removed from the data. This could had been done for the dataset which was used out of which the authors filtered samples containing the two keywords related to covid and mask to make sure that the data is from humans.

Pipeline and comment:

As I had a video in german language I first tried the model comments in germany language. By viewing the Semantic textual similarity matrix I was a bit confused as the similarity for all words was quite high. By taking a closer look I was capture for cell which were a bit darker I was able to spot same words patterns wich ware appearing in the two compared sentences like:

"liest du eigentlich auch mal was du schreibts ?"

"niemals, schonmal mal was von wahlfälschung gehört ?"

By taking a look on the top similar comments I was not able to find the comments are sorted based on their length. Is I had only around 700 comments I viewed all and noticed that on the top comments with one or 2 full sentences appeared and towards the bottom of the list very long comments and only one or two-word comments were grouped.

Because I was curious If the model performs better for english language I tested the model also with comments on a english video.

For the Sematic textual similarity matrix it showed a clear difference. I assume the model was trained with english words and is less confused/ more certain with english sentences. I think we can used this model as a kind of pre-filtering and sorting of the comments.

My Notebook.

[JuliusBro]

Read and comment

This paper seems to be in precautionary mode as the team does not make use of platform specific features such as retweets and likes and also performs some cleaning on the data before further processing. Things that were not grasped or mentioned are things like bot manipulation or even manipulation of Twitter's sampling that we read about in a previous assignment. This mode of inquiry also misses further networks of agreement trough disregarding the like and retweet mechanisms that make it relatively easy to gauge how many people agree with something

Pipeline and comment:

The model seems to not be useful for smaller comment-chains. My video only had around 400 comments so there were very little similarites, as is evident by one comment being "Shame. There is no asteroid, it is man made agenda to sow fear among the populace. Yet many believe it 😔" and the most similar comment being "Dude, are you kidding yourself? The planet is HUGE compared to your body. Your body will succumb to this cancer at a rate faster than the planet. Think about the obvious" with seemingly no similarities. The model could be quite useful in the project for the influence of the alt-right to determine the amount of use of alt-right dogwhistles, since these can be presumed to appear relatively often and then use this to classify the users accordingly.

link to notebook

[Moritzw]

In which ‘mode of inquiry’ (Marres and Moats 2015) is the research project and paper operating? What are the issues that are silenced / cannot be grasped by this mode of inquiry?

The mode is most likely the precautious mode. The researchers used the sample data set (1% free api) from Twitter and filtered Posts by keywords. Firstly they filtered all posts loosely connected to covid-19 that were collected over a 5 month period. Afterwards those were again filtered for keywords related to covid-19 and mask-wearing. But while they cleaned the data of URLS and unrelevant characters for easier processing, they did not filter again for bots or possible bias (corporate advertisements, possible manipulation of the sample set) in the data-set. This filter method excludes all posts that do not mention covid-19. In the designated timeframe from the researches it is highly likely that the mentioning of facemasks alone is related to covid-19. A great number of relevant comments is likely lost in the analysis due to this filtering method. Furthermore they discarded all platform specific elements like retweets, followers or likes, despite this being integral features of Twitter. Therefore any influence those features have on the discussion is lost. This is a major issue that is being silenced, since the number of retweets and the likes of such tweets influence the sentiment of the followers who receive them.

What can the model do well? When does it fail? The model does well in finding similar comments, but the heat-map feature does not show much usefulness. The labels are only readable when using truly small sample sizes, at the same time, only by using large sample sizes does the heat-map show if there are large clusters of similar comments or if the comments are mostly individual statements. Sadly due to the labeling issue one can't determine which comment has many similar comments and which comments do not.

How can you use it in your project? It makes it easier to find how many people write similar comments, which helps to determine if a specific comment is a prevalent opinion distributed over the comments or just a minority, one that might even have gamed the sites algorithm to be represented more strongly in the topmost comments. In my test, both with random comments and with specifically selected ones, the selected similar comments made sense and I could not find any explicit false positives.

Jupyter Notepad

[alexkhrustalev]

Was done by Alexey Khrustalev and Anastasiia Todoshchuk

1 From reading the paper “Unmasking the Conversation on Masks: NLP for Topical Sentiment Analysis of COVID-19 Twitter Discourse” it was hard to define certainly, which of the approaches mentioned in the Marres and Moats paper were applied to the current research. The easier way is to say, which approaches were not applied and to confirm this position. Firstly, according to the paper “Tampering with Twitter’s Sample API” by Pfeffer, Mayer, and Morstatter, Twitter API provides researchers with unrepresentative data sampling, which obviously can easily lead to wrong results of an analysis, and it can be treated as the feature of the mentioned API and Twitter itself. There were no mentions of this feature in the current research and these characteristics were not taken into account. That’s being said, we can’t say for sure that the researchers used a truly precautionary approach or, at least, to try to use such an approach was not perfect.

2 The model shows good results in finding similar comments. However, it has some problems with the heat map, as the model fails to plot the similarities in this way. I can imagine using this model to find similar comments and to analyze them. Since our goal is to detect bots, it may be possible to find the bot’s comment and to find similar comments using the model. It could limit human interaction, as it would be sufficient to find only 1 bot-comment using a human. Then, the process could be repeated iteratively and all the similar comments could be treated as the bot’s comments. There can be another way to use the model. It may be not that easy to say for sure that a comment was written by a bot. To prove the assumption the model can be used: if there are a lot of super (open question: how to define the border?) similar comments - they could be written by the bots.

https://github.com/Einnmann/khrustalev_todoshchuk_assignment4/blob/main/Assignment-4_Language_model.ipynb

[budmil]

1. The paper „Unmasking the Conversation on Masks: Natural Language Processing for Topical Sentiment Analysis of COVID-19 Twitter Discourse“ is operating in the precautionary mode of inquiry (Marres and Moates). Such conclusion is clear from the methodological approach the research is following: they directly strive into the data, without paying attention to possible platform-based specifics (likes, RT, Quoted RT) or ‘irregularities’ such as bots (both human or programmable). This way of research is good in a sense that it takes only the pure data into account, but it also has some flaws - meaning that the selected tweets could be poisoned by bots or company accounts (in this case big public health institutions for example). Also the lack of platform specific points of view could affect the results. Another interesting thing is that in this paper Twitter’s API is praised as objective, while in paper (Pfeffer, Jürgen, Mayer, Morstatter) from 2018. we could see totally different thing. It could be that Twitter improved the algorithm in these two years.

2. When it comes to the work at the notebook, it is worth mentioning that the pipeline clearly has some space for improvement. The code would be useful in a research project as a broad view of comments topics, especially in videos with high number of comments (thousands). The heatmap did not produce what I expected. One comment that I singled out is: "you failed to mention what are they spraying. total lies a sellout is the worse thing anybody can be. do a real research before you post nonsense." Btw, the video being observed has almost 5000 comments.

[travela]

1. My classmates above have found the consensus that the paper was conducted in the precautionary mode, which is hard to for to argue against and by the principle of exclusion I, too, end up at this conclusion. Clearly the authors did not mention anything about platform specific influence on the controversy and they did in fact heavily filter their data in advance. This should exclude the empiricist and affirmative approach. However, what they did filter out were retweets as well as alleged off-topic data, which is not connected to detecting bot activities. So there is no compelling argument to be made for a precautionary mode as defined in Marres and Moats 2015. But maybe I am still lacking a clear understanding for what the different approaches stand for and whether they define a cover for all possible approaches.

2. As one would expect the model works well if there are similar words in two different comments and it works best if those words are arranged in a similar order. This is the case e.g. when people are referencing quotes from the video, so in such cases the pipeline usually found highly related comments. On the other hand when I tried it for some random comment containing the word "you", I received a ton of other comments with the word "you" in them, but naturally the content of the comments was not related at all. I haven't decided on a project topic yet, so I am yet to find out where these tool will come in handy.

Notebook in my fork repository

[yuxin16]

1. Reading Assignment

I suppose that the authors did the research mainly from a precautionary mode since the consideration of platforms features and bot detections are not undertaken by researchers (as far as I recognized in this paper).

The researchers mainly focus on key phrases regarding "COVID 19" and "wearing masks" but ignore user interactions focused on certain topics (clusters and sub clusters in this paper). Therefore, the researchers although discovered the concentrated discussion around certain topics, but failed to discover the dynamics of discourses around them. The platform features of twitter which may influence the dynamics of discussion stays currently unresearched. For example, retweets without replying or number of likes show somehow agreement on certain comment, but when this features/interactions get ignored, the cluster outcomes maybe inaccurate/in-appropriate for user-trend discovery.

Additionally, the clustering method applied in this research seems to be hard clustering which ignore the interaction/fuzzy areas between different clusters. Some comments may belong simultaneously to disjoint clusters and may display links/relationships between different discussion clusters. Furthermore, the analytical biases from different sources (data bias, sampling bias etc.) can also rise problems in research.

2. Comment Analysis Assignment

The most problems I encountered with was the unexpected long execution time and I faced the ResourceExaustedError serveral times. It is horrible when dealing with large set of data and get my laptop crashed (But maybe, it is not the problem of the algorithm, rather of my computer capacity). The model works well for explicit/short sentences but seem how misunderstood/misinterpreted long sentences with complicated structure. However, it did well in discovering neighbors (find similar comments) if the given comment is clear in its wording and meaning (not emotionally with ironic factors or so). I think this model can be well used to find similar comments if we want to research on single comment and explore the discussion around similar opinions, but how and whether it can be used into our project, depends totally on the research question and research design.

3. Link to Pipeline xinyu_Assignment_4

[Francosinus]

1. Reading

The aim of this analysis was to study COVID-19 and mask related Tweets and detect specific topics and sentimentality trends within this content over time. But the authors in this paper only focus on the content of the tweets: tweets about COVID-19 and wearing masks, but don't differentiate between the two topics. Retweets, number of likes or any other interactions were not taken into account, which are typical features of this platform. Thus, I think the precautionary approach describes it the best. In addition they use the 1% Twitter API which can result in bias due to data manipulation (as we previously learned).

2.Pipeline

I already worked a bit with similar methods previously, hence I know that this can be computationally expensive. My laptops memory is definitely not good enough to perform a large analysis (this is why I could only use 10.000 out of 40.000 comments and reduced the batch size). But in general the model performs well and in case of my comment example it could find a lot of similar comments. A common procedure also is to fine tune a model based on the data, but that is beyond the scope of the course. I like the idea of working with embeddings, but for an analysis with large datasets it's computationally impossible for me to use this. But if for example a topic modeling is performed beforehand (or some additional filtering of the text) this could reduce the memory consumption. OR I have to use my work laptop :D

3. Link

Assignment Link

[milanbargiel]

1. Reading

The research paper uses a precautionary mode of inquiry. Social media and platform-specific artefacts such as URLs and non-punctuation characters are removed from the dataset to analyse the content with the use of natural language processing, clustering and sentiment analysis.

In a precautionary approach, the social media platform's influence on the controversy itself is not taken into account. Contextual information is deleted from the dataset, to make it more substantial. Nevertheless, especially this contextual information could give information on the nature of the controversy and its dynamics. Do people in real-life express the same level of polarization as found in the data set or is it just a social media phenomena that take place on Twitter solely? Who are the main actors in spreading content on COVID on Twitter and how do they influence the discourse?

2. Pipeline

The model does well in finding similar comments that are based on the same topic and contain the same buzzwords. The expressed opinion is not distinguished as easily and you find comments that deal with the same topic but express totally different views on the subject of climate change. Furthermore, the model works much better with short statements like „I think this guy is getting paid by oil companies".

The project that I will work is the analysis of discourse dynamics in social media. The model can help to identify topics or themes that are reoccurring and have a high potential to spark a specific outcome of a discourse. It would be interesting to find out what topics are most hazardous. Furthermore, it could be interesting to use sentiment analysis to analyze what feelings are expressed for videos that are scientifically grounded or based on conspiracies and compare those two.

3. Link

Notebook

[jtembrockhaus]

Reading assignment: Sanders et al. drew conclusions about public attitudes towards the preventative measure of mask usage during the COVID-19 pandemic. Since the study was focused on topic clustering and sentiment analysis, a detection of possible non-human tweets produced by bots was not performed. The authors of the study payed most attention on the content of the discussion itself and not on the plattform-specific dynamics and features of Twitter. Retweets, the number of likes and threads were not considered, which are key features of twitter that can definitely influence the course of debate. The general approach of Sander et al. Can best be assigned to the precautionary mode of inquiry. The lack in the analysis of platform-specific effects can be seen as an issue of the approach, since it decreased the representativeness of the overall findings.

Model analysis: The model in general seem to work well. However, comparing a small data set of only ~500 comments (that also tend to be very long) made it very difficult to find similarities in the comments. Therefore the heatmap could not present very meaningful results, which should change when investigating a larger amount of comments. Here, I only tried a test video and didn’t want to redo the entire analysis due to the time consumption regarding the word embeddings. If I will use the pipeline in the final project or not very much depends on the actual task. In any case, the pipeline is a good starting, that can be expanded depending on the planned analysis.

Notebook Link

[xixuanzh]

1 Reading assignment

• Unlike other fellow students, I would like to assume that the paper of Sanders et al. features an empiricist approach. Though the scholars cleaned the data before data analysis, the paper focuses on conversations on mask usage during the COVID-19 pandemic. It did not examine the negative participation of digital platforms through bot and influentials detection (precautionary mode) or take platform-specific aspects into account (affirmative mode).
• The findings are limited to the content of the related discourse on Twitter. The users are neglected in this context. The possibility of unauthentic behaviors caused by bots or political campaigning during the covid which would largely change the narratives on this topic is not considered.

2 Language Model

• The model did well in finding similar comments with related keywords. Yet, it did not offer satisfactory performance when dealing with lengthy comments with multiple arguments and ideas.
• The methods in this paper is highly relevant for the project discourse dynamics, from clustering to sentiment analysis; all these parts are widely used in the field of related research.

3 Link https://github.com/FUB-HCC/seminar_critical-social-media-analysis/blob/master/Pipeline/xixuan_zhang_assignment_4/Assignment-4_Language_model.ipynb

[DanielKirchner]

Assignment 1

The most fitting mode is the precautionary mode, since twitter platform specifics, such as the use of likes, retweets, hashtags were not (directly) analysed in this paper. The main focus is only on the meaning/intention/sentiment of the text in the tweets that were processed. The tweets are collected through the 1% API, which we know not to be a representative sampling method for social media analysis, which might overly represent bot accounts or people exploiting the techniques shown in last weeks assignment. There was also no filtering done to prevent parts of the data set (around 1.000.000 samples) to be from users which may spam posts (bots) with the filtered out keywords.

Assignment 2

Because of an automatic python update to 3.9 on my system conda broke down completely for me today. I can't activate an environment anymore and tensorflow isn't available for 3.9 yet. I will address this in our meeting tomorrow.

Working now: Notebook