notconfusing / wiki_econ_capability

using economic relfections method modelling to determine wikipedia editor and article rank
0 stars 3 forks source link

explain broad impact #5

Closed notconfusing closed 9 years ago

notconfusing commented 10 years ago

Explain the research's contributions/broad impact. The paper only cites one CSCW article. I don't have a problem with this, but I think this is indicative that the paper needs to better situate its contribution to the CSCW discipline. What can we do with this new way of modeling article quality and expertise? What are its implications towards Wikipedia and/or other similar online collaborative editing systems? How do the optimal alpha and beta parameters and the model inform socio-technical theories of Wikipedia practices? What would be the next steps (beyond tweaking the algorithm)?

        R1.1. Introduction: It is a bit surprising that the topic of
       information cascades is introduced referencing only two
       unpublished works (one of them under review). It would be
       great if the authors could also include other pointers to
       the literature of team organization and complexity.

        R1.2. The authors claim their work to be “the first attempt to
       quantify the value of collective contribution environments
       from the collaboration structure alone”, but I think that,
       at least in the domain of Wikipedia, the work by Luca de
       Alfaro and collaborators should be credited. A lot of work
       in the field of trust and reputation management tried also
       to look quality of contributions, as well as some works by
       Ulrik Brandes on collaboration networks.

       R1.3. The related work section would use a more thorough review
       of the literature. Some words should be spent on the
       connection between public goods and team production for
       example. In Microeconomics, for example, this is studied as
       a coordination problem. There is also a large literature in
       software engineering about software teams and quality.
       Another work that looked at the growth of Wikipedia pages
       and quality is the one by Huberman and Wilkinson.

       R1.4. I would tone down the claim that measuring quality is
       impossible. This is simply not true. Experts are able to
       evaluate non-code artifacts, and even opinions by
       non-experts can be aggregated into meaningful signals. OTOH,
       it seems a good idea to talk also about unpredictability of
       quality (cf. the work by Watts, Dodds and Salganick). Also,
       natural language can be evaluated with a lot quantitative
       metrics (some of which you actually use as ground truth
       later in the paper), so please tone down also the language
       there.

       R1.5  The point about number and quality is well taken. To this
       end, I would suggest to look at the work by Scott Page on
       diversity and teams.