Study: Islands of Knowledge

[westwood846]

As part of my Master's Thesis on programming expertise distribution in open source software, I analyzed the repositories of some popular NPM packages for so-called Islands of Knowledge.

Islands of Knowledge are a metric for the Bus Factor of a project (which states how many developers can get hit by a bus before the project is incapacitated). An Island of Knowledge exists when single a developer alone has expertise in an programming domain. A programming domain is an area of expertise about certain programming tasks, like working with databases or web APIs.

To find Islands of Knowledge, I implemented a data mining process to count how many times each developer used each module and combined the numbers for modules in the same domain (for example the IO domain contains, among others, the modules fs and path). "Modules" in this sense also means global objects like console.

I analyzed keystonejs/keystone and found the following potential canditates for Islands of Knowledge:

Developer	Domain	Modules/Objects	% of uses
webteckie	BROWSER	object-assign, watchify	73%
Jed Watson	HTML	keystone-utils, marked	90%
jstockwin	FRAMEWORK	keystone-nightwatch-e2e	99%

To validate my results, I would like to ask the keystonejs/keystone developers a few simple questions:

1. Do you think that the listed developers have most of the knowledge of the listed programming domains?
2. Do you think that keystonejs/keystone may be in trouble if the listed developers leave the project?
3. Does keystonejs/keystone have some characteristics such as detailed documentation so that the loss of the listed developers would be less troublesome?
4. Do you think that the assignment of modules to domains shown above is reasonable?

Thanks in advance for your collaboration,

Marco Brack
Software Languages Team
Koblenz, Germany
http://www.softlang.org/

[jstockwin]

@turbopope Sounds like a really cool and interesting project!

keystone-nightwatch-e2e (KNE) is currently used to automatically test keystone for continuous integration purposes. It is something myself and webteckie (who is also in your list) have split off from the main keystone repo so that it can also be used by any other apps using keystone to test those apps.

It is therefore correct that KNE would likely fail without myself and webteckie. However, this would not bring down keystone itself, since keystone would survive without KNE. (In fact, I get the impression it is moving away from using KNE anyway).

Do you have the stats for keystonejs/keystone itself as a module?

I hope that helps to some extent, and if it's brought up any additional questions then please go ahead!

To specifically answer your questions, for my keystone-nightwatch-e2e entry only: 1) Yes, but also webteckie is probably on a similar level to me. 2) No. KNE would be, but keystone itself would be fine (although it would need to adapt how it does continuous integration) 3) No. There should be docs on it, but I don't think there are really. 4) Yes.

[westwood846]

Thanks for the reply!

Unfortunately I had to find out that this issue violates the GitHub ToS and I'll have to close it. Please apologize my misbehavior.

[export-mike]

@turbopope interesting this is against ToS what does it violate?

[westwood846]

I contacted support because I got flagged for the offense (they have un-fagged me now that I closed the issues). I was told:

Your account was flagged for opening issues on other users' repositories for the purposes of drawing attention to your research project, which is prohibited by our Terms of Service.

I assume this refers to https://help.github.com/articles/github-terms-of-service/#i-advertising-on-github, where it says:

We reserve the right to remove any advertisements that, in our sole discretion, violate any GitHub terms or policies.

So even though I think that whether my survey qualifies as spam is debatable, it's their decision :-/