langcog / wordbank

open repository of children's vocabulary data
http://wordbank.stanford.edu
GNU General Public License v2.0
64 stars 10 forks source link

add warning on norming reports page #94

Closed vmarchman closed 9 years ago

vmarchman commented 9 years ago

A box including the following text should appear on the norming reports page.

The norming reports provided by Wordbank are dynamically generated and reflect currently available data. For English and Spanish, the values provided by Wordbank are based on a different set of participants and are estimated following a different computational procedure than the norming values published in the User’s Guide and Technical Manual (Fenson et al., 2007; Jackson-Maldonado et al., 2003) and applied in the CDI Scoring program (Marchman, 2013, mb-cdi.stanford.edu/scoringdb_p.htm). We recommend that Wordbank-generated values be used for research and descriptive purposes only. For clinical applications, users should refer to the norms and guidelines published in these manuals (available through Brookes Publishing Co, www.brookes.com). For clinical applications in other languages, users should refer to the appropriate manuals and norming information, as listed on the contributors page.

mcfrank commented 9 years ago

Thanks! Are you ok with a summary of this living on the main page and the full text on the "more details" pull down?

Mike

On May 23, 2015, at 8:14 AM, vmarchman notifications@github.com wrote:

A box including the following text should appear on the norming reports page.

The norming reports provided by Wordbank are dynamically generated and reflect currently available data. For English and Spanish, the values provided by Wordbank are based on a different set of participants and are estimated following a different computational procedure than the norming values published in the User’s Guide and Technical Manual (Fenson et al., 2007; Jackson-Maldonado et al., 2003) and applied in the CDI Scoring program (Marchman, 2013, mb-cdi.stanford.edu/scoringdb_p.htm). We recommend that Wordbank-generated values be used for research and descriptive purposes only. For clinical applications, users should refer to the norms and guidelines published in these manuals (available through Brookes Publishing Co, www.brookes.com). For clinical applications in other languages, users should refer to the appropriate manuals and norming information, as listed on the contributors page.

— Reply to this email directly or view it on GitHub.

vmarchman commented 9 years ago

yes, but it would depend what the summary would say. I would not want the main message to be hidden too deeply.

Upon further reflection, I would like to amend the complete statement as follows: (edits welcome, especially in the first sentence about the curve-fitting procedures. should we say something here about exactly what those curve-fitting procedures are?).


What do I need to know about norming reports in Wordbank?

The norming reports in Wordbank are dynamically generated applying sophisticated curve-fitting procedures that make a particular set of assumptions. Morever, Wordbank norming reports reflect currently available data culled from different sources that may not be identical to those included in the norming samples for a particular instrument. In particular, users should be aware that the values provided by Wordbank for English and Spanish are based on different participants and are estimated following different computational procedures than the values published in the User’s Guide and Technical Manual (Fenson et al., 2007; Jackson-Maldonado et al., 2003) and applied in the CDI Scoring program (Marchman, 2013, mb-cdi.stanford.edu/scoringdb_p.htm).

In general, we do not recommend that Wordbank-generated norming values be used for research or clinical purposes in which the goal is to evaluate children's performance in reference to an established normative standard (e.g., determining the mean words produced percentile of a group of children in a study; classifying whether the performance of an individual child falls below the 10th percentile). For these types of applications in English and Spanish, users should refer to the norms and guidelines published in the manuals for those languages (available through Brookes Publishing Co, www.brookes.com). For similar applications in other languages, users should refer to the appropriate manuals and norming information, as listed on the contributors page.

mikabr commented 9 years ago

Feel free to edit the text of shiny_apps/vocab_norms/details.md on branch iss83 to change the text of the disclaimer/explanation.

mcfrank commented 9 years ago

done - but that branch doesn't play nice with dev/prod right now so I can't see the markdown render. nevertheless, I'm closing this issue for now.