jrising
commented
1 year ago Thank you for these helpful comments! I respond to each one below.
Section: Producing Results
could you potentially create a more descriptive title. "Producing Results" is quite generic
We have now called this section "Some pointers when performing regressions", which is much more descriptive.
I don't quite understand the learning outcome -> who is "offering"? The verb should refer to what the student will be able to do by the end of the chapter.
Good points. The learning outcome is now rewritten as "Avoid common pitfalls when running regressions and plotting results."
is there a python implementation for the error term?
Yes, there is an R implementation of the Conley HAC standard errors. I have now added a tab with this link.
Section: Hands-On Exercise, Step 4
note that I wasn't able to test the code in this section because I had an error in Step 3.
Thank you for looking at this. In addition to fixes in the underlying library, which hopefully would let you run step 3, we have also added an R implementation of step 3 using a newly developed library.
Again, I would appreciate a python implementation of this section.
Thank you for pushing us to do this. We have now added this. It did require some hand-built code to do the state-specific trends and to calculate the confidence intervals of the dose-response function, but hopefully this will be helpful for future students.
in the "Running the regression" sub-section, what do you mean by "state trends"? Why do you use state trends versus county trends? Why have county fixed effects but not state fixed effects? It would be helpful if you explained your choices. Perhaps, you could build in some sensitivity tests for students to do on their own to justify the choices - for example, what happens to the results if they don't include FEs or use different ones? What about the linear and quadratic temperature terms? Does having both improve the results? How do they know which model is more robust (not sure if you use an adjusted R^2 for this type of thing)?
These are good questions.
-
We have clarified that "state trends" meant "state-specific trends" and added a written-out specification.
-
It would certainly be reasonable to consider county trends. We have now added a discussion to help students decide on the right level and structure of trends.
-
State fixed effects are actually not necessary here. They are colinear with the county fixed effects and so would be dropped.
-
We have now included a discussion of the kinds of sensitivity tests, and comments about using R^2 in combination with looking at the resulting fit.
-
The inclusion of linear and quadratic temperature terms makes sense given what we know about the effects of cold and hot temperatures on mortality. This is explained now in the text.
Here is the much-extended discussion in the tutorial now based on your comments:
Let's run our central regression, relating death rate to temperature. In keeping with the literature, we will assume that weather-related mortality is a u-shaped function. That is, both cold and hot temperatures cause increased mortality. We also assume that these effects are convex, so that more extreme cold and heat will produce a more-than-linear increase in mortality. The simplest relationship that has these features is a quadratic. Refer to the functional forms section for further considerations.
For fixed effects, we use county fixed effects, to account for unobserved constant heterogeneity, and state-specific trends, to account for gradual changes at a larger scale. While the high-resolution fixed effects are necessary and standard, the choice of trends can be more subtle. Here are a few points:
- The high-resolution and more flexible the better, as far as identification is concerned. For example, we might consider county-specific trends, higher-order polynomials, or cubic splines. These better isolate weather shocks, which are the core of the identification strategy.
- At the same time, flexible trends capture the variation necessary for statistical analysis. For example, a state-by-year fixed effect would leave very little variation at the county level, because counties weather is correlated within each state.
- The right balance between these depends upon the other drivers affecting the dependent variable. It is useful to graph the timeseries for individual units and for groups of units (e.g., every county within a state). Trends should be imposed whereever there is non-weather-related drift in the dependent variable, and their resolution should be fine enough so that the values for all the contained regional units are drifting in the same way.
- Where reasonably researchers can disagree about the right trend specification, generate tables that show several specifications. Typical sensitivity tests include more flexible fixed effects and functional forms and the inclusion of other weather variables. The R^2 values will help you understand how much of the variation you are capturing, but it is important to look at the resulting fit to decide if the regression is doing what you expect.
In our case, we see gradual shifts which are captured fairly well by state-level trends. We also only have 10 years of data, but with more data, more saturated fixed effects should be explored.
Here is our specification: $$M_{sit} = \beta_1 T_{it} + \beta_2 T_{it}^2 + \gamma_i + \delta_s t$$ for mortality in state $$s$$, county $$i$$, year $$t$$.
Note that since we do not control for precipitation, temperature in this case includes correlated effects of rainfall. As a result, it is not ideal for future projections.after the figure, it might be useful to describe what is shown. How do you want students to interpret the figure? (Don't forget to revise C to deg C in the figure x-axis as well).
Great point. We have now added this paragraph:
The estimated dose-response function suggests that counties with an average temperature higher than about 2 deg C observe increases in mortality when the temperature rises and decreases in mortality when it falls. Counties with very cold average temperatures see the opposite-- for them, warmer temperatures reduce mortality.
Since baseline mortality rates are different in each county, the dose-response function can only tell us about the effect of changes. So, for example, relative to the mortality expected on a 20 deg C day, a 30 deg C day is projected to result in 0.3 additional deaths per 100,000, equivalent to deaths attributable to all natural disasters in the US.Section: Suggestions for work organization
second note: I don't think you introduced the term "vector data" in the pervious GIS section.
We have changed this to read "geospatial polygon data" to be clearer.
kls2177
ks905383
This Chapter nicely wraps up the tutorial. Just a few comments below.
Section: Producing Results
This section is out of my area, so I only have a few comments.
Section: Hands-On Exercise, Step 4
Section: Suggestions for work organization