Discovering higher-level Patterns (E3) - DeDeo et al 2018

[HyunkuKwon]

Post questions about the following exemplary reading here:

Barron, Alexander TJ, Jenny Huang, Rebecca L. Spang, and Simon DeDeo. 2018. “Individuals, institutions, and innovation in the debates of the French Revolution.” Proceedings of the National Academy of Sciences 115(18): 4607-4612

[toecn]

I find Barron et al.'s paper fascinating. In particular, I'm interested in their measure of novelty using KLD. A project that I have in mind uses tweets from politicians; more precisely, we are interested in seeing innovation in political rhetoric on Twitter for a set of politicians over a ten-year period. I wonder if KLD would be a good measure for us, given that we only have tweets as units of text (vs. speeches of other units with more data). Should we be cautious about applying this measurement in a context with small text units? Could we add the smaller units to make for larger units? What other measurements could work? How do we conceptualize time in this context and operationalize it?

[yushiouwillylin]

My question is a methodological one. In many of these influence-related papers, there are always some parameters that need to be chosen, like in this paper, we have to choose number of topics or the speech window to calculate novelty or transience. Do institutional details usually fit in the selection of these papers? Or do researchers usually just pick the most fit parameter to gain the best accuracy?

[Raychanan]

I was surprised that I didn't see any data results reported on accuracy, did the authors feel this was unnecessary? Why didn't they report this (these) metrics?

Also, I think the amount of data they used may not be large enough. So I still have some reservations about this study.

[romanticmonkey]

In the first part of the result, Barron et al. claimed that novel speeches are "unexpectedly influential," but in the second part, we saw a slight contrast where low-novelty high-resonance persons exist. Hence I came to wonder: is it really the novelty that makes speeches/ideas influential? Or is it some other properties, present in the high-resonance speakers of both high and low novelty.

[jinfei1125]

It's really interesting to see the combination of Kullback-Leibler Divergence and topic modeling to study the novelty, transience and resonance of texts over time. But this article says:

novel speeches were unexpectedly influential. We quantify this with “resonance,” the imbalance between novelty and transience (see Materials and Methods). Resonance, the quality of at once differing from the past and leaving traces on the future, increases with novelty

Also, in the appendix, we can find that Resonance = Novelty - Transience

So, given this strong relationship, I wondering if it is necessary to study the relationship between resonance and novelty while we already study the relationship between transience and novelty like the following figure shows. I apologize if my question sounds very immature, but I am really curious about the answer. Thanks in advance!

[k-partha]

It thought this was an excellent read and a fantastic example of how CSS insights can bolster traditional social science analysis, though, as some people above, I have some reservations about the 'bluntness' of metrics used (How closely is underlying data tracking the usual conception of 'novelty' and KLD? Are there better metrics than KLD?). The dynamics between the 'left' and the 'right' here point to an interesting interplay of discourse - I am wondering if we can replicate the same patterns across countries/time - this would really support the paper's findings and also raise important questions in pol-sci. Have these patterns been replicated elsewhere?

[jcvotava]

This paper's conclusion seems to be that revolutionary French leftists... used novel, revolutionary rhetoric. In contrast, conservative/royalist Frenchmen tried to preserve existing rhetoric and language. Given a like 2-sentence historical description of the French Revolution, this seems immediately obvious and not very interesting. I'm wondering: how could you use topic modelling in a case of retroactive historiography like this to make more surprising interventions? Or is topic modelling just the wrong tool here?

[zshibing1]

The number of topics k=100 used in latent Dirichlet allocation seems arbitrary. How do we choose the number of topics in general?

[william-wei-zhu]

French revolution is significant because it caused a major shift in the source of legitimation. In other words, people obey not to an arbitrary ruler (the king), but to a set of rules and principles. I am interested in learning about how the early debates of the French revolution impacted such a transition.

[Bin-ary-Li]

As important as it is, French Revolution was hundred years ago and democracy has been evolving (devolving?) ever since. I wonder if the model demonstrates (or can be used to demonstrate) any difference between now and then?

[ming-cui]

After reading several articles using a large size of text content, I begin wondering if there is publishing bias in the results. Different groups of researchers may utilize different modeling techniques and subjectively choose different parameters. I have not seen any influential articles trying to replicate others' findings. Maybe academics emphasize innovation and originality too much. Is researchers' discretion a concern when we evaluate an article's conclusion?

[egemenpamukcu]

My question is about the use of topic modeling and LDA in this paper. Why did authors use topic modeling instead of just looking at the divergences of speeches? Do they use LDA just as a way of representing text and providing input for divergence calculation? When would such an approach make sense?

[mingtao-gao]

What are metrics that can or should be used when evaluating topic modeling?

[RobertoBarrosoLuque]

It was not clear to me how the authors optimize the number of topics they use for LDA and how they keep track of the different topics throughout their text on the french revolution? Specifically how their measures of novelty ties in with their use of topic modeling.