Discovering higher-level Patterns - Orientation

[HyunkuKwon]

Post questions about the following orienting reading:

Timmermans, Stefan and Iddo Tavory. 2012. “Theory Construction in Qualitative Research: From Grounded Theory to Abductive Analysis.” Sociological Theory 30(3) 167–186.

[wanitchayap]

The abductive framework is very interesting, and I think it is a valuable reminder for us, computational social scientists, that we cannot do works solely from data but we also need good theories.

My question is about applying "the role of method" in the framework of computational social science. To me, revisiting and alternative case, as elaborated by the authors, might be harmed by the computational method. For example, through data analysis and data reconstruction, we may miss some small but important elements that we could have revisited by close examinations if it were through a qualitative framework. Or data analysis and data reconstruction may change our raw data such that it favors one theory over the others (through which variable we chose to extract, which model we chose to use, etc.), and hence we cannot fully take advantage of the alternative casing. On the other hand, I can see the computational method helps us defamiliarize the phenomenon by the same data analysis and data reconstruction. Reducing the big phenomenon into lower dimensions allow us to see things we cannot at the unit level. I guess my ultimate question is that, do all three methods the authors propose appropriate in the computational framework? Do you think computational social science research should integrate some qualitative part? Or it should remain to be purely computational but still under the abductive framework?

[WMhYang]

This paper reminds me of the orienting reading of week 1 by Prof. Evans. For that week I asked the question how could we believe that our unsupervised methods are yielding new patterns in contents. The answer by Prof. Evans was that we could use simple models and exogenous data to evaluate the unsupervised machine learning models. Currently I am rethinking this question in the framework of this paper. Could I treat the model evaluation work as revisiting and defamiliarizing because in this step we are making sure that we fully understand what is going on behind the model and what is unexpected or uninterpretable by existing theories? Should there be a further step alternative casing that we develop and modify existing theories or patterns to rationalize the results yield by the models?

Reference: Evans, James and Pedro Aceves. 2016. “Machine Translation: Mining Text for Social Theory” Annual Review of Sociology 42:21-50. DOI: 10.1146/annurev-soc-081715-074206

[ihsiehchi]

My concern is again similar to Mint's, the way I would phrase it is that the efficacy of the three methods is compromised as one moves from a textual framework to computational analysis. But unlike Mint who focused on revisiting and alternate casing, I thought defamiliarization was the one that would most likely be rendered redundant. The reason is that computational analysis actually lowers the familiarity threshold for one to conduct meaningful research. An example I thought of was the Old Bailey paper, as one reduces court transcripts to a list of words with the highest frequencies, even an analyst without substantial content expertise / domain knowledge (there must be a better term for what I'm thinking) has something novel to say. To a researcher who mainly employs ethnography as her investigative strategy, computational analysis would be a source of defamiliarization. But to a computational analyst, isn't the very act of converting texts to vectors the destruction of opportunities for defamiliarization.

[nwrim]

I have a myriad of questions right now (most of them will be almost purely meaningless to anybody except me in this class), but I think the most important two are:

• Similar to what @wanitchayap and @ihsiehchi discussed - I think the "wriggle room" in quantitative data is substantively smaller than qualitative data (importantly, this does not necessarily mean that quantitative works are more scientific than qualitative works, whatever that means). Therefore I feel that Revisiting the Phenomenon, Defamiliarization, or Alternative Casing might not be as effective as from a qualitative perspective. From a more experimental research point of view, it is often the case that we try to "rule out" alternative theories by controlling for variables rather than allow space for the alternative theory to kick in when revisiting the data. In this process, it is rarely the case that the original data provides a way out - you can fight quantitative data only by quantitative data. This fundamentally means that even more data should be collected to address the concern. For example, if I had data that I claimed to be proving theory A - then the abduction comes in, and say "theory B can also explain the data". However, since we have data specifically aimed at answering theory A, for this abduction to work we need to collect data related to B. At this point, I feel like this is just a loop of induction, rather than abduction. Long story short - I feel like abduction strategy, at least in the form of revisiting data, is fundamentally restricted in a quantitative sense. I would love to hear Prof. Evans' or anyone else's thoughts on the issue.

• I thought a lot about recursion while reading the paper. Basically, the process is:

  def abduction(data):
  theory = form_theory_from_data(data)
  data = revisit_data(data, method="revisiting the phenomenon")
  data = revisit_data(data, method="defamiliarization")
  data = revisit_data(data, method="alternative casing")
  if not explain_data_by_theory(theory, data):
      abduction(data)
  else:
      return theory

  This arises a natural question - do we ever get in the else clause? Can we ever know if we reacehd our base case and our theory is correct? Can we make strong claims about any issue if abduction is the base method in investigation?

[iarakshana]

Maybe I wasn't able to decipher it from the paper, is there a way to test abductive theories - with deductive theory - it must be true if the underlying premise is true so if you can disprove the underlying theory, the deductive theory isn't true I guess and similarly with inductive we can disprove it if we find a contradiction? - With abduction theory, we are suggesting that we're revealing something we didn't see before so I'm not sure how we would test/ evaluate that?

[linghui-wu]

Similar to the point mentioned by @ihsiehchi, I have concerns about defamiliarization. It sounds like reflexivity in interviews where individuals reflect on their socioeconomic background when conducting interviews. Being reflexive helps both interviewers and interviewees to be more aware of the context behind the topics. So with that comes my question, it seems more natural to talk about defamiliarization in qualitative researches but how can we implement this concept in quantitative studies where we heavily rely on computational techniques to realize analysis?

The second point of @nwrim is also of great interest. Generally speaking, is there a framework for understanding when we’ve reached the “base” case and how this method adjusts or concludes if there is ultimately no theories to construct?

[timqzhang]

I have the similar question with @nwrim . As the new theory is unknown at present, how do we decide the "maturity" of the theory? In other words, when is the point for us to stop such an iterative process? It will be stopped finally, as the data input must be finite (at least for certain period), either with new theory out or nothing to return, but the latter one is not the goal for this abduction method. Could we instead just select some "candidates" of theories and then make a comparison among them by further assessment? It could to some degrees avoid the situation that no new theory come out when the data is exhausted.

[Yilun0221]

In considering various theoretical perspectives, the researchers focus on rethinking and defamiliarizing and considering different definitions in this abductive method indicates that diversity of thinking is important. My question is, apart from collaborative relationships among scholars, would participatory methodology contribute to analyzing a broader population group?

[tianyueniu]

It is mentioned in the reading that 'The switch from induction to abduction thus requires a gestalt switch in which the theoretical background is foregrounded as a way to set up empirical puzzles'. I am not sure I fully understand the concept proposed. How should we decide when to turn the switch in social science research?

[harryx113]

The paper mentioned abductive analysis as "a qualitative data analysis approach aimed at generating creative and novel theoretical insights through a dialectic of cultivated theoretical sensitivity and methodological heuristics". Given that insights and heuristics are hard to quantify, how to ensure the consistency of such method?

[DSharm]

In principal, I definitely understand where this idea of abductive reasoning is coming from - the idea of being open to anomalies and surprises and allowing them to shape theory makes sense. In practice, however, it feels like we could easily end up mining the data to tell a story we want it to tell, finding spurious relationships, or focusing too much on anomalies and missing the big picture. Echoing some of the other concerns above, how can we avoid these pitfalls?

[Lesopil]

From what I understand, one of the essential arguments from this paper is that good research needs to be well grounded in other good research, with researchers relying on a lot of previous theoretical work to begin their own analysis. What if there isn't that much previous experience in your area of interest with computational methods? How well do the patterns identified by qualitative researchers align with the patterns that we can identify computationally?

[jsgenan]

Abductive analysis, from my perspective, is the interaction between empirical findings and theories. As the examples of Vaughan and Timmermans show, the puzzles from data could be well fitted into a grounded theory framework. The reading also mentions that anomalies could require the development of tentative new theories built on inductive conceptualization. Yet there will always be inconsistency between theory and data. My question is, from your experience, what is the ideal end of an abductive cycle?

[liu431]

I am wondering what're the general critiques for abductive analysis among sociologists. Is it accepted as the guiding principle now?

[lyl010]

I am wondering the to which extend the following statement can be true: If the analysts start with raw data he will end up initially with a substantial theory.(page 169) I am wondering because there are lots of noises in raw data, and I am thinking the author is taking about a sharp insight. Another field of study: anthropology focus on the behaviors in the long time. What is sociology difference with anthropology on the abstraction of a theory?

[shiyipeng70]

Obviously the concept and application of abductive analysis is very significant. Personally, I feel confused more than a couple of times when I was reading a paper or an academic presentation that what was the point in proving a new hypothesis with the basis of previous theoretical framework, especially during a time when research begins to cram into nuances. Big ideas and abstract theories are intentionally avoided under the guidance of "Vienna circle". However, Peirce writes that a hypothesis worth pursuing has to be decided in abductive analysis and one particular direction has to be chosen within an universe of possibilities. I wonder how can we get rid of all the biases and charisma of the previous inductive analysis and draw a distinctive scope?

[pdiazm]

The paper certainly makes a strong case for abduction, but I'm not convinced about the author's claims surrounding the background theories that the researcher is expected to leverage as he / she analyzes the data. I think it's impossible to completely remove bias for one background theory or theories over the rest - our brains will naturally try to fit a theory to our knowledge (however limited this knowledge might be).

[timhannifan]

For Timmermans and Tavory:

The authors provide a strong argument for the use of abductive analysis in bridging the gap between theory and empirical evidence. The abductive framework challenges the idea that the data should speak for itself, and how one might approach an unsupervised learning project.

How can we situate unsupervised learning within the abductive process? Are these compatible approaches? Which methods are more useful in this context? It seems like Bayesian non-parametric topic models, as described by David Blei, allow for the “surprise factor” in that these models “can observe previously unseen topics”. In addition, dynamic modeling seems to satisfy Timmerman’s criterion for time-series analysis.