Study 1 Design

[ndphillips]

Game Structure

• Reduce total # of trials to 25. This would make all choices more immediately relevant to the goal.
• Make it very clear that options do not change over time.

Goal conditions

1) No Goal 2) Goal with Bonus on top of individual earnings 3) Goal with all-or-nothing bonus - If you don't reach the goal, you earn nothing

Note: The goal should be moderately difficult to reach.

Distributions - Always 2 options

Note: Absolute values of parameters are just for illustration

Condition	A	B
Same EV, different variance	N(mean = 3, sd = 2)	N(mean = 3, sd = 4)
Different EV, High EV = low variance	N(mean = 3, sd = 2)	N(mean = 2, sd = 4)
Different EV, High EV = high variance	N(mean = 2, sd = 2)	N(mean = 3, sd = 4)

1) 2 Options with same EV: A) High Variance, Medium EV, B) Low variance, Medium EV 2) 2 Options with different EV and negative risk-return relationship: A) High variance, Lower EV, B) Low Variance, Higher EV 3) 2 Options with different EV and positive risk-return relationship: A) High variance, Higher EV, B) Low Variance, Lower EV

Notes

[mdsteiner]

Proposal Study 1 Design

Game Structur

• As you mentioned above, use fewer trials (25)
• 8 or 10 games
• Make it clear that the environments are static, i.e. they do not change
• Use a cover story

Goal Conditions

Only two conditions due to power issues. Since in the simulations only a sufficiently hard goal lead RSF to outperform EV maximization, use a hard goal of 100

• No Goal
• Goal, Bonus is only paid if the goal is reached

Distributions - Always 2 Options

Condition	A	B
Same EV; Different Variance	mean = 3; sd = 2	mean = 3; sd = 8
Different EV, High Variance = low EV	mean = 3; sd = 2	mean = 2; sd = 8

Participants

Aim for 92 participants per condition. This makes a total of 368 participants. This way we should have enough power to detect small to moderate effects and have somewhat reliable results.

Program study

Implement the task in javascript and code the r part as efficient as possible. Maybe this way we can ensure that the task runs smoother with smaller lag and hopefully fewer crashes so that we could be as close to 92 participants per condition for the analyses as possible.

[ndphillips]

• Looks good, But I would really like to have three environments. That is, including a different EV, high variance = high ev.

• I propose 75 participants per condition for a total of 75 participants x 6 conditions = 450 participants. With 10 games per participant, this will give us tons of data.

• You'll have to show me your power calculations later

• I'm also happy you'll try to implement some of the code in java script. Good luck!

[ndphillips]

What should the goal be?

If the goal is 100, and people have a ~ 20% chance of reaching the goal, they might become frustrated and give up. What are the solutions?

• Drive home that it's difficult to reach the goal, they should not expect to reach the goal in each game but should always try.

[ndphillips]

Change goal to 100, and adjust means / standard deviations to adjust game difficulty. 100 should be easier to process than 90

[mdsteiner]

When we use the following environments we should still get the wanted effects (although rsf now only very slightly outperforms ev in terms of proportion of goals reached in environment 2). In the simulation the goal is reached 38% of the times. But this is mainly driven by environment 1...

Condition	A	B
Same EV; Different Variance	mean = 4; sd = 2.5	mean = 4; sd = 11
Different EV, High Variance = low EV	mean = 4; sd = 2.5	mean = 2.5; sd = 11
Different EV, High Variance = high EV	mean = 2.5; sd = 2.5	mean = 4; sd =11

[ndphillips]

I'm a bit concerned that the standard deviation of the risky option is so large that people won't be able to detect its mean very easily. Can you try playing the game yourself and see how difficult it is to distinguish environment 2 from environment 3?

Actually, try quantifying difficulty of each environment with two statistics:

1) What is the probability that a single sample from the high EV option is larger than a single sample from the low EV option? 2) What is the probability that the high EV option gives a positive outcome? What about the low EV option? What is the relative likelihood of the two?

[mdsteiner]

I think when you play the two environments right after each other it is relatively easy to see that in one you're getting nowhere with the safe option and in the other you can come close to the goal by using the safe option. Still I think environment 2 will be the hardest for participants to figure things out... Here are the numbers (note that A and B are ordered different in environment three here compared to the above tables; The code is in 06_issue4.R):

Condition	A	B	a > b	a positive	b positive	a positive / b positive
Same EV; Different Variance	mean = 4; sd = 2.5	mean = 4; sd = 11	0.50	0.95	0.64	1.47
Different EV, High Variance = low EV	mean = 4; sd = 2.5	mean = 2.5; sd = 11	0.55	0.95	0.59	1.60
Different EV, High Variance = high EV	mean = 4; sd = 11	mean = 2.5; sd =2.5	0.55	0.64	0.84	0.76