mdsteiner
commented
7 years ago Ok I added an analysis script. I tried to describe all the predictions etc. I think it would be worth discussing the script before preregistering it. The simulation shows all the predicted results, so now only the participants have to behave as simulated...
Here's the idea: run a simulation of agents behaving in the exact conditions for Study 1. Then, write code that does the main statistical analyses for these data. Once we have code and predictions we like, then we can lock these into the pre-registration.
Here are the steps I envision:
1) Simulate agents performing study 1. In the goal conditions, make everyone behave RSF. In the no-goal conditions, make everyone behave EV max. You could assume everyone uses an e-greedy strategy, or that each uses a soft-max strategy with different parameters. Feel free to make any other simplifications you need to emphasise the size of the effects. Because we won't make any pre-registered predictions specifically about modelling, we don't need to worry about the specific parameters too much.
2) Define our most important behavioral predictions that come out of the simulation, with a focus on comparing the goal and environment conditions. For example:
Trial level
Probability of selecting high-variance option given that one is below 100 points (probably should be higher for the goal condition than the no-goal condition)
Probability of selecting high-variance option given that one is below 100 points (probably should be lower for the goal condition than the no-goal condition)
When the RSF and high EV model make different choice predictions, the choice prediction from the RSF strategy will be more accurate than the EV strategy for the goal condition. However, in the no-goal condition, the EV strategy will make better predictions. You would need to do this by, for each trial, for each participant calculate the predicted choice for the RSF strategy and the predicted choice for the EV strategy on the next trial.
Game level
proportion of high variance options chosen overall.
Probability of reaching 100 points. (Probably more people reach 100 points in the goal condition than in the no-goal condition).
Absolute points earned. For example, we might predict from the simulation that people earn more points in one environment than another, and people in the goal condition might actually earn fewer points than the no-goal condition.
These are just my intuitive predictions from what we’ve talked about in the past. However, if the simulation doesn’t show one or more of these, we’ll have to discuss more).
In terms of the specific statistical test, mixed effects regression seems to be a good bet (with random intercepts for participants, games, and trials). For binary variables (DV = choose high var or choose low var), you'll need to do a logistic variant.
As I think you’ve already done all of the necessary simulation code, I hope this should not be too much work to organise. If it is, let me know