In the EDA reading, you have one paragraph that I was reflecting on:
“This is problematic because any activity that involves data but lacks a clear motivation is doomed to be unending and unproductive. Data science has emerged precisely because our datasets are far too complex for us to understand directly; indeed, I would argue that the job of a data scientist can be summed up, in part, as a person who identifies meaningful patterns in our data and makes them comprehensible.”
My question on this is whether or not a clear motivation is necessary for a preliminary analysis/EDA? A data scientist might “explore” the data in the process of conducting an EDA (whether directed or undirected); won’t what matters be what they synthesize and then present to the stakeholder?
Me:
This is definitely the branch I'm exploring walking out on, and I recognize that in taking such a strong position, I'm sure I'm not quite right. But given our students are coming from the opposite extreme, I'm feeling out the more extreme position on the other side.
With that said, I am finding it pretty compelling. To some degree I think it depends on your definition of "clear motivation" and how narrow-precise one interprets that. I don't expect people to dive in to their data with most of their paper already written and a need for nothing but the values to plug into their tables (if they're the research paper-writing sort); but I do think you need a clear sense of what matters in the sense of "what outcomes are problem relevant? what independent variables might you have leverage to manipulate (if your goal is to be impactful)?
Put differently, what you synthesize and present to the stakeholder is the conclusion or answer to a question, but the metric for whether something is substantive significant comes from the stakeholder's problem.
But your point is well taken and I'll keep thinking about it.
Kyle:
That perspective makes perfect sense. Erring on the side of overcorrection from many students’ current default of approaching an EDA like a treasure hunt without a map is reasonable. I agree it hinges around “how narrow-precise” the interpretation of what a “clear motivation” is. If there’s some wiggle room to make that a bit less narrow-precise, then the paradigm you present here feels very well-composed.
Kyle:
In the EDA reading, you have one paragraph that I was reflecting on:
“This is problematic because any activity that involves data but lacks a clear motivation is doomed to be unending and unproductive. Data science has emerged precisely because our datasets are far too complex for us to understand directly; indeed, I would argue that the job of a data scientist can be summed up, in part, as a person who identifies meaningful patterns in our data and makes them comprehensible.”
My question on this is whether or not a clear motivation is necessary for a preliminary analysis/EDA? A data scientist might “explore” the data in the process of conducting an EDA (whether directed or undirected); won’t what matters be what they synthesize and then present to the stakeholder?
Me:
This is definitely the branch I'm exploring walking out on, and I recognize that in taking such a strong position, I'm sure I'm not quite right. But given our students are coming from the opposite extreme, I'm feeling out the more extreme position on the other side.
With that said, I am finding it pretty compelling. To some degree I think it depends on your definition of "clear motivation" and how narrow-precise one interprets that. I don't expect people to dive in to their data with most of their paper already written and a need for nothing but the values to plug into their tables (if they're the research paper-writing sort); but I do think you need a clear sense of what matters in the sense of "what outcomes are problem relevant? what independent variables might you have leverage to manipulate (if your goal is to be impactful)?
Put differently, what you synthesize and present to the stakeholder is the conclusion or answer to a question, but the metric for whether something is substantive significant comes from the stakeholder's problem.
But your point is well taken and I'll keep thinking about it.
Kyle:
That perspective makes perfect sense. Erring on the side of overcorrection from many students’ current default of approaching an EDA like a treasure hunt without a map is reasonable. I agree it hinges around “how narrow-precise” the interpretation of what a “clear motivation” is. If there’s some wiggle room to make that a bit less narrow-precise, then the paradigm you present here feels very well-composed.