What can we measure about knowledge transfer and natural language?

[andymeneely]

We've started discussing this, so it's time to create an open-ended question.

Is there some sort of natural language analysis that we can conduct that will gives us insight into what these code reviews are doing in terms of knowledge transfer? Are developers learning from each other through these collaborations?

Maybe we need to conduct a survey of these developers? We might learn something from that.

[bspates]

Possible Research Questions

• Can we track knowledge transfer through a given developers vocabulary over time?
  • Good candidate to include social graph theory in the study
  • Knowledge transfer here would be measurable in the interaction of experienced developers with novices, and the spread of vocabulary.
• Can we correlate a given file's vulnerability over time to the use of security words in code reviews containing said file?
• Can we categorize code reviews based on context of the review and patch sets involved?
  • Categorization could be based on a number of factors:
  • Sentiment of the reviewers
  • experience of the reviewers and of the reviewed
  • code context
  • vocabulary used (technical, security)
• Can we track knowledge transfer from owners to new developers over time?
  • Create a corpus of an owners code commits and compare to new developer's code commits over time and in relation to interaction with file owner in code reviews.

[bspates]

Postgresql supports natural language searching: http://www.postgresql.org/docs/9.1/static/datatype-textsearch.html All we have to do is change the field type of out text blobs