make explicit criterion for broad utility

[davidlmobley]

@dmzuckerman - your changes basically look good to me.

I'm not sure I agree with Michael that we necessarily want to require them to work on a number of different approaches or systems; I think this should be addressed in assessment of the prsubmission letter and review and is covered by the "broad utility" criterion. Specifically, I can imagine, say, a well constructed PMF study that fails in a very informative way that will have broad utility/implications for a large number of researchers, even though it is the result of a "single experiment".

On the other hand, I agree with you, @mrshirts , that we don't want this to be just a dumping grounds for one-off studies that are just "Well, we tried this one thing one time and it didn't work but we'd like to publish it somewhere so we get something out of it, even though it's unlikely to be of any real benefit." But I think that's likely to be covered by the "broad utility" clause and not something we need to explicitly state, partly because it's hard to state it in a way which would prevent us from taking some high quality and informative "one off" studies while allowing us to reject low quality, uninformative "one off" studies.

[dmzuckerman]

I agree with @davidlmobley; I think the broad utility requirement is a valuable but sufficient addition. I don't think we need to add additional hoops @mrshirts. It's clearly explained in the author instructions that a Lessons Learned article is real science ("Quite simply, a "Lessons Learned" manuscript is a regular scientific article and hence should be prepared with the standard high level of care and documentation. ... Data analysis should conform to best practices."). So I think that is sufficient. As with all the article types, the burden will be on the editors and reviewers.

[mrshirts]

Perhaps the suggestion for multiple studies and/or multiple systems be a suggestion for how to know if it's broad utility.

I guess in general, if Dan is being sufficiently selective on these, then it's OK. I would suggest at least keeping track of what is coming in and eventually making more explicit criteria, so it's easier for editors to know what passes the criteria. More information about the types of manuscripts we want can only help, though perhaps it will take a year or two to figure out how to express it clearly.

[davidlmobley]

Sounds good.

[dmzuckerman]

@mrshirts check it out now. I agree this should be helpful.