Simulate divergence between RSF predictions and maximizing

[ndphillips]

Questions

1) Can we show via simulation how a risk sensitive foraging strategy will outperform a straight EV maximising strategy? 2) In which statistical environments do the two strategies have different performance? 3) What aggregate descriptive differences do we see in the strategies (e.g.; option switching, proposition of high variance choices).

Create a function that takes the following arguments:

• mean.v a vector of option means
• sd.v a vector of option standard deviations
• trials number of trials in the game
• goal the magnitude of the goal
• nsim number of simulations
• strategy Which strategy does the agent use? RSF or EV maximising

The function should return statistics such as the probability of reaching the goal, mean points earned, option switch rate (etc.)

Strategies

EV maximising: Soft-max rule taking only means into account RSF: Soft-max taking into account likelihood of reaching the goal.

• All strategies use a simple updating rule
• Could also use an e-greedy choice rule instead of softmax

Notes

• We can use this function to a priori determine stimuli that should maximise the difference in predictions between an RSF model and a maximising model.

[mdsteiner]

It seems only if the high variance is huge (I used 15 with high EV of 5 and low EV of 2 with var 1) we see the shift in preference for risky options for above vs. under the goal for "ev" and "rsf" strategy.

Above Goal

Under Goal

[ndphillips]

Got it. But the difference in EVs in this example is quite large (2 vs 5). What if you changed the EVs to say 2 and 3? I'm guessing that you could then decrease the magnitude of the variances and still get the effect.

[mdsteiner]

Ok I ran some more simulations.

• high EV only outperforms rsf if the goal is relatively easy (e.g. 70 with means of 3 and 2 and 25 trials), for high goals (e.g. 90 with same parameters as above) rsf outperforms high ev strategy for all environments
• as you mentioned above, with means of 3 and 2 we can set the higher variance to 6 or higher to get the effect

So I think we should make sure to have a sufficiently high goal and maybe a variance of 7 or 8 to have an effect. I didn't play with the learning and exploration parameters though.

[ndphillips]

Very good! Which selection model did you use? egreedy or softmax? If you haven't already, I would make sure that the results hold for egreedy. As we discussed before, the problem with softmax is that the temperature parameter is sensitive to the scale of the expectations (which differ between EV and RSF). If we can get the results with egreedy (whose parameter is independent of the expectation scale), I think that should be sufficient for now.

But later, we'll probably need to do a more complete simulation using different softmax parameters

[mdsteiner]

I used egreedy to prevent the problems we had last time when we ran the simulations with the softmax.