Trial of image based or text context

[markwhiting]

From slack:

In another new paper, Yejin Choi et al deal with context in a kind of nice way: they use images as stimuli. https://arxiv.org/pdf/2310.10418.pdf We’ve always had a bit of trouble with context because our claims each only have a few words — certainly not the 1000 an image might be worth. We could certainly try providing context cues with images to see if they make people responses more consistent. @Amir — if it is quick to set up, we could try to run a small trial, say 150 people with the same 20 statements each with 2 different sets of context images (and a control with no image or some kind of provably unrelated image) randomly assigned. So we get 50 of each condition who have answered the same statements under different context frames. We would then check how the context frames appear to impact the variability of responses.

We amended this with text context which should be clauses that can be added to statements to modify the context. The images should provide a similar idea to the text context.

[markwhiting]

This is all set to go. But we might need more images and a higher level experiment design.

We also need to decide if it is worth doing, or perhaps when we should do it.