tjbrailey
commented
3 years ago 3- Complete 4- Complete 11- Complete
Open tjbrailey opened 3 years ago
tjbrailey
commented
3 years ago 3- Complete 4- Complete 11- Complete
Enumerated below are various critiques to incorporate into the main manuscript:
I would encourage you to consider the potential for selection effects. That is, the type of power-sharing arrangements agreed upon (these are the ones you are most interested in) could depend on the context and power balance of the groups as well as third-party guarantees at the time agreements are made. These might, in turn, influence the effects as well. In other words, you will have to separate whatever independent effects the power-sharing arrangements have from the effects that these other variables have on the outcomes of interest. In particular, changing distributions of power or variations in the commitment of third parties could influence the outcomes in ways that might either reinforce the effect of power-sharing or go against it. These issues might, in turn, account for some of the instability of the parameter estimates.
Beginning with Table 3, it is noted that We see that the presence of other consociational provisions exert a negative and statistically insignificant effect on levels of social trust in a state, though this becomes positive and significant at the ten-percent level when we interact this variable with the regional autonomy measure (p. 36), and later argue that the results constitute convincing evidence in support of my hypotheses (p. 39). However, this is not quite right. When interpreting results involving interaction terms, you need to consider the lower-level terms in specific ways depending on the nature of the data and the research question. Given that your two regressors of interest here are binary, to analyze whether the combination of regional autonomy and other provisions have a positive or negative effect on trust, you need to add together the coefficients on regional autonomy, other provisions, and their interaction. In this case, that would be -30.83 - 12.28 + 29.00 = -14.11. This would mean a decrease in trust, though not as much a decrease in trust as if only regional autonomy were implemented without other provisions (-30.83), or only other provisions were implemented without regional autonomy (-12.28). In other words, putting aside issues of statistical significance, the evidence suggests that these various provisions (regional autonomy and other provisions) are associated with less trust, though that lower trust is alleviated somewhat if the two (regional autonomy and other provisions) are implemented together. Another way you can view this is that you have four types of countries. 1: Those with neither (a) regional autonomy nor (b) other provisions. 2: Those with only (a). 3: Those with only (b). 4: Those with both. The results suggest that those with neither (a) nor (b) (i.e. no power sharing) are associated with the highest levels of trust, those with both (a) and (b) the second highest level, then followed by those with only (b), and then only (a). In other words, having only regional autonomy is associated with the lowest levels of trust, and even when you combine it with other provisions, it is still associated with decreased trust relative to the condition of having neither. These point estimates thus provide evidence counter to your Hypothesis 1 and Hypothesis 2. Though taking statistical uncertainty into account, it is more likely that the evidence is simply unsupportive of your hypotheses.
The same problem applies to the interpretation of Table 4: Similar to the results in our OLS on social trust levels, the interaction effect between regional autonomy and other consociational provisions serve to reduce conflict to a statistically significant degree. The Cox model indicates that these variables reduce conflict by a factor of 47% (p. 38). For the same reasons already laid out above, this is not quite right. To analyze whether the combination of regional autonomy and other provisions have a positive or negative effect on conflict, you need to add together the coefficients on regional autonomy, other provisions, and their interaction. In this case, that would be 0.38 + 0.58 - 0.64 = 0.32. This would mean a hazard ratio above 1, that is, an increase in conflict, though not as much an increase in conflict as if only regional autonomy were implemented, or only other provisions were implemented. Again, this is counter to your hypotheses. As an additional note, Models 2 and 3 in Table 3 look strange in the sense that all of the coefficients are the same across the models. This could be the result of a code bug, or else it might be the result of the interaction term being linearly dependent with some of the variables that are not displayed, most likely country fixed effects. The results for your third dependent variable, democratization, do appear to be in favor of your hypotheses. However, as the paper itself notes, the use of this as a dependent variable is potentially problematic from a measurement/definitional perspective. It is commendable that this issue was addressed in the robustness checks section. With respect to that robustness check (Table 6), however, it is highly surprising that the coefficient on regional autonomy changes so dramatically while the coefficient on other provisions does not change at all when you add the interaction term. The only reason I can think that this would occur (other than a code bug) is that, as already alluded to above, the interaction term is linearly dependent with the country fixed effects, which would mean adding the interaction term simply causes a country fixed effect to drop out but does not actually change the underlying model (it simply slightly reparameterizes it). You can verify this by checking whether the predictions or residuals across Models 2 and 3 are identical or not.
In addition, some more minor comments: p. 21: H2: Ethnically diverse states that employ segmental autonomy provisions along with other provisions of power-sharing, either through peace accord or constitutional reform, will experience reduced intergroup tensions, prolonged peace, and democratization. - H2 could be written in a clearer manner that makes its testable implications distinct from H1, along the lines of the employment of additional provisions of power-sharing moderate the effect of segmental autonomy provisions on...
p. 30: The unit of analysis, scope of the sample, and data structure should be more explicitly delineated before getting to variables. For instance, it is not explicitly stated that the data structure is country-year.
p. 30: On the conflict DV, how can you be sure that regional autonomy is not granted as a precondition or negotiated demand to end hostilities? A decrease in conflict as the result of a minority group having their demand for autonomy satisfied is quite different than the granting of autonomy putting in place the underlying sociopolitical conditions to keep conflict at bay over the long run.
p. 30: More clarification on the independent variable would be useful. Since the unit of observation is country, but the level of analysis for autonomy is subnational, it is slightly unclear how you code countries. Is it simply that a country that has any autonomous regions is coded as a 1? This is my assumption here, but it is not made as explicit as it could be.
p. 31: The use of polity score as a control variable and a dependent variable presents a conceptual tension. Using it as a DV indicates that you believe segmental autonomy has an effect on the polity score. That then means that in the other regressions where other DVs are employed, the inclusion of polity score is the inclusion of a post-treatment variable. This is an issue because insofar as the polity score (or democracy level) has its own effect on conflict and/or trust, the inclusion of polity score as a control in those regressions will then attenuate the estimated effect of segmental autonomy on conflict and/or trust by netting out the portion of the effect that is propagated via democracy level. In sum, it results in an estimated effect of segmental autonomy on conflict and/or trust that is either biased or needs to be interpreted more narrowly.
p. 35: robust ordinary least squares (OLS) regression - Not sure exactly what you mean here. The term robust can mean many things with respect to a regression analysis. Are you referring simply to the estimation of standard errors? Or are you employing a process that automatically drops outliers? Or something else?
p. 35: In the equations, the matrix X should have a vector of coefficients associated with it.
p. 36: Epsilon should not be included in the Cox hazard model.
p. 39: The high RMSE value and R2 value explains fifty percent of the variance in the relationship between autonomy and social trust, and so when accounting for country and year fixed-effects and our control variables, these core provisions explain a good amount of the underlying patterns in the data. - Your R2 values explain how much of the variation in the outcome is explained by the regression as a whole, not specifically by your autonomy or other provisions variables.