fsolt / dcpo_dem_mood

1 stars 0 forks source link

Validity of DCPO #18

Closed sammo3182 closed 3 years ago

sammo3182 commented 3 years ago

R1: Although intriguing, no evidence at all is presented to support the claim (repeated throughout) that this method is "superior" to Claassen's (reference is made to an unpublished working paper). Indeed, it would appear from the short summary of this model that it adds additional parameters to Claassen's model. Yet more complicated models may overfit the data and are not necessarily more accurate. I propose that evidence supporting the superiority of this method be supplied, or alternatively, the authors just focus on replicating Claassen's models.

R4: The author should provide more information on the differences between the models employed by Claassen and those that she/he employs (perhaps in an appendix). We need more information on (1) how uncertainty is integrated into the models of Claassen, and (2) the DCPO models.

fsolt commented 3 years ago

TBH, and I recall saying this before, we really don't need DCPO in this paper. We can very easily consider it yet another robustness check, mention it in the text briefly, and push it off to the appendix. On DCPO, R2 also---somewhat snidely--- writes, "This MS disputes this finding on two main grounds: that (for several reasons) Claassen's measure does not take full advantage of available survey data, and that Claassen's analysis does not account for measurement error. Since the first critique strikes me as valid but fairly modest in importance, I concentrate on the second, which receives the bulk of the attention in the MS as well." I think that that's a faux-generous reading of what we wrote (we're quite explicit about what is important), but we opened the door to it by including DCPO as a second, weaker argument. We should just make the one. big. point. and minimize anything that distracts from it.

Doing so will win us back at least a third of a page or so, at the top of page 5 (which can all go in the appendix), plus a few words here and there, all of which we may well end up needing. Presenting the "more data" dataset with Claassen Model 5 makes our argument much cleaner.

Have I convinced you @sammo3182 @Tyhcass ?

fsolt commented 3 years ago

Keep it in per @sammo3182

Tyhcass commented 3 years ago

@fsolt @sammo3182 Double check, regarding DCPO, we will put analysis on Appendix, is that correct? I think we have DCPO estimates from corrected and corrected and expanded data, which are we used in our previous version, is that right?

fsolt commented 3 years ago

Agreed on both counts. I will confirm that the source data for the corrected-and-expanded DCPO matches the most recent version; if not, I'll re-run on Argon.

fsolt commented 3 years ago

Closed by 3fe47db86bf6d4ebbfccbab2c2056a5d82205180