bensoltoff
commented
7 years ago - Lots of attrition with the screening questions. Was the experiment too difficult, or were the MTurkers just especially incompetent at this experiment?
- On plot 4, why do the true sum means differ? Wasn't it the same data presented in both graphs, so shouldn't that be the same? Or is this a feature of the bootstrap process? - nevermind, just checked the code and it refers to the actual mean value in the sample, not the true value in the data
- Seems like a tradeoff between bias and variance. Common in stats, though odd in the graphical interpretation. Not sure I've seen this in the literature before. How do you decide as the researcher to balance that tradeoff?
- Really basic but interesting research question motivating the study
I came up with the design/idea for the survey, created the data/visualizations for the survey, conducted analysis on the results, and cleaned up a draft of the write up.
Jonathan Camacho conducted the survey on Qualtrics/MTurk and wrote up the initial draft of the assignment submission.
This assignment was alright. I'm really glad that I had screening questions in the submission that I could filter the results with, otherwise it would have been difficult to discern "good" from "bad" data. Another option we might have looked at was a bonus for getting within a certain range of the true value, but that would have been a little more difficult to set up.
The main takeaway we got was that dodged charts have less precision but more accuracy, and stacked charts have more precision but less accuracy (with less accuracy potentially being because of some round 100's number bias).