search

HEGSRR / OR-Replicability-in-Geography-Survey

BSD 3-Clause "New" or "Revised" License
0 stars 0 forks source link

Q9 + Q11 Coding #7

Closed Peter-Kedron closed 6 months ago

Peter-Kedron commented 1 year ago

We may want to code Q9 and Q10 together. They were designed as text responses to our division between study and phenomena characteristics, but respondents seem to have treated them as one. Specifically, there are reasonably rich answers to Q9 that also discuss characteristics of phenomena. Then researchers say see Q9 in their Q11 answer, and the response rate drops dramatically for Q11.

Peter-Kedron commented 1 year ago

@josephholler I reviewed these two questions to try and pull any ideas we did not cover in the corresponding MCQ Q8 and Q10. Below is a list of concepts I saw. Those that are in bold were mentioned with some frequency across respondents. I did not do any formal counting or coding at this point, as I wanted to cross check how we wanted to handle this one. My lean is to sprinkle these in a paragraph around the formal presentation of Q8 and Q10, as opposed to doing some formal counting.

Q9+Q11 Coding Possibilities • Effects of sample composition • Whether Initial Sampling was Random (Well designed?) • Artifact Accessibility • Need for specialized/expensive equipment • Funding • Perceived Importance of Results • Location Accessibility (Remote Sites), Logistics • Pressure to Publish Novel Results • Infrequency of events (e.g., 1000 year flood) • How well-studied location already is • Familiarity with Theories Used • Surprising Result • Unique samples (e.g., fossils) • Use of concepts that invite multiple operationalizations • Popularity of the topic • Occurrence of irreversible events that alter a site/phenomenon • Differences in context

Two other very common responses were 1) Qualitative researchers used this question to reject replication outright. They claimed that it was not possible for one of several reasons – 1) Theoretical nature of the work, 2) Positionality, 3) Rejection of positive science, ...

2) A number of respondents were not clear about what both Q9 and Q11 question were asking.

josephholler commented 1 year ago

I agree. There's a relatively small sample of interesting responses and they can be used to add context & discussion to the quantitative responses.

Peter-Kedron commented 1 year ago

Great. When you eventually review my draft of Q8 and Q10 in the overleaf draft please do that with this in mind. If you insert where you think valuable, we can look at when we line edit and review together. I'm hoping I can get a full draft together and maybe we can sit down for a day again in Sept do the review like last time.