Ideas for extensions

[eriktaubeneck]

I played around with this and it's really cool! I had a couple thoughts.

I wonder if there would be a way to build in some kind of constraint? Perhaps for every round you have 10 tokens to spend to "exploit" and you have to chose which campaign they go to, and the system spends another 10 tokens uniformly to "explore"? I realized that in the absence of some kind of budget constraint, the 'correct' answer is just to spend money on any campaign that gets you conversions. Or rather, any campaign that gets you conversions at a cost-per-acquisition lower than your profitability target.

I am guessing, however, that CPA is probably too difficult to simulate though and then how do you even think about simulating a user's profitability target. Thus making conversion rate a decent proxy.

Also I thought it would be easier to make decisions if the numbers at the decision screen were presented as a rate/ratio and started playing wasn't clear to me if I was supposed to assume every campaign had the same number of impressions or not although then. Presenting it as a conversion rate would also allow you to take into account the spend choices adjusting the number of impressions from the previous experiment round if you want to.

I could probably get some privacy engineers at snapchat interested in playing the game for a while too so let me know! Who it is easy to recruit (privacy and software engineers) might be different than who you want to recruit in a perfect world (marketers?)

Originally posted by @jgoodknight in https://github.com/patcg/meetings/issues/155#issuecomment-1777573960

[eriktaubeneck]

Also I thought it would be easier to make decisions if the numbers at the decision screen were presented as a rate/ratio and started playing wasn't clear to me if I was supposed to assume every campaign had the same number of impressions or not although then. Presenting it as a conversion rate would also allow you to take into account the spend choices adjusting the number of impressions from the previous experiment round if you want to.

This makes sense, and it easy to add. Thanks for the suggestion!

[eriktaubeneck]

I am guessing, however, that CPA is probably too difficult to simulate though and then how do you even think about simulating a user's profitability target. Thus making conversion rate a decent proxy.

I'm not sure what distribution to use to compare the different CPMs from in order to simulate CPA, but open to suggestions!

[eriktaubeneck]

I wonder if there would be a way to build in some kind of constraint? Perhaps for every round you have 10 tokens to spend to "exploit" and you have to chose which campaign they go to, and the system spends another 10 tokens uniformly to "explore"? I realized that in the absence of some kind of budget constraint, the 'correct' answer is just to spend money on any campaign that gets you conversions. Or rather, any campaign that gets you conversions at a cost-per-acquisition lower than your profitability target.

I'm open to thinking around this, but my primary concern about moving in this direction is that it makes it harder to understand at which point the participant is making essentially random decisions.