Registration: prior experiment replication

A preregistration based on the As Predicted template.

Data collection. Have any data been collected for this study already?

Yes, we already collected the data. No, no data have been collected for this study yet. It's complicated. We have already collected some data but explain in Question 8 why readers may consider this a valid pre-registration nevertheless. (Note: 'Yes' is not an accepted answer.)

Hypothesis What's the main question being asked or hypothesis being tested in this study?

Do the results of "A framework for quantifying individual and collective common sense" replicate? (see paper https://doi.org/10.1073/pnas.2309535121 and materials and prior registration https://osf.io/9kxt2)?

This is an exact replication with a minor caveat: instead of answering 50 random statements, participants will answer 15 statements in a design point condition (drawn from the exact statements from the prior work) — this allows improved PQ common sense evaluation.

We aim to evaluate the same main analysis and expect the following similarities:

Plainly worded fact like statements about everyday physical reality will have the highest commonsensicality
Variation in commonsensicality is higher between statements than between people.
In PQ common sense, although a small number of people may share a large number of beliefs and a large number of people may share a small number of beliefs, the volume of cliques between these extremes is proportionally small. We will compute this by averaging design point level PQ common sense calculations across design points within this experiment

Dependent variable Describe the key dependent variable(s) specifying how they will be measured.

Metrics defined https://osf.io/9kxt2/

Individual and Statement commonsensicality
PQ common sense

With the addition of:

The definite integral of PQ common sense across design points

Conditions How many and which conditions will participants be assigned to?

Conditions are design points in the space of possible statement types, sampling 15 statements per design point:

6 binary knowledge properties: Social vs Physical, Everyday vs Abstract, Figure of speech vs Literal language, Normative vs Positive, Opinion vs Factual and Knowledge vs Reasoning.
knowledge domain question with 13 values: General reference, Culture and the arts, Geography and places, Health and fitness, History and events, Human activities, Mathematics and logic, Natural and physical sciences, People and self, Philosophy and thinking, Religion and belief systems, Society and social sciences, and Technology and applied sciences.

2^6 13 7 = 832 total design points.

Not every design point has enough statements in the prior corpus, so we are using the 67 design points which have at least 15 statements each from that corpus. In situations where the number of statements is larger than 15 we select the 15 clearest statements, via a clarity feature present in the prior data.

Analyses Specify exactly which analyses you will conduct to examine the main question/hypothesis.

We will compare core metrics of commonsensicality across grouping variables with the results in https://osf.io/9kxt2/. We will report 95%CI to evaluate similarity between results.
We compute the PQ common sense curve (and clique array) for each sampled design point. We will aggregate these design point-level PQ common sense curves to a population-level representation to compare with prior results.

Outliers and Exclusions Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations.

We will exclude data of participants who provide incomplete responses or fail to meet attention checks in the survey tool.

Sample Size How many observations will be collected or what will determine sample size?

No need to justify decision, but be precise about exactly how the number will be determined.

We aim to sample at least 30 participants per design point.

Other Anything else you would like to pre-register?

(e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?)

This is part of an ongoing examination of design point level analysis of common sense.

Name Give a title for this AsPredicted pre-registration

Suggestion: use the name of the project, followed by study description.

Design point level replication of Quantifying Common Sense

Type of study.

Class project or assignment Experiment Survey Observational/archival study Other:

Data source

Prolific MTurk University lab Field experiment / RCT Other:

Watts-Lab / commonsense-platform