Timezone and Time-of-day Variance in GitHub Teams: An Empirical Method and Study

[timm]

Timezone and Time-of-day Variance in GitHub Teams: An Empirical Method and Study

https://github.com/researchart/swan17/blob/master/pdf/swan2017xyz.pdf

Open source projects based in ecosystems like GitHub seamlessly allow distributed software development. Contributors to some GitHub projects may originate from many different time zones; in others they may all reside in just one time zone. How might this time zone dispersion (or concentration) affect the diurnal distribution of work activity in these projects? In commercial projects, there has been a desire to use top-down management and work allocation to exploit timezone dispersion of project teams to engender a more round-the-clock work cycle. We focus on OSS in GitHub, and explore the relationship between timezone dispersion and work activity dispersion. We find that while time-of-day (TOD) work activity dispersion is indeed associated strongly with time-zone dispersion, it is equally (if not more strongly) affected by size of the project team.

[timm]

AUTHORS: Important. Do NOT reply till all three reviews are here (until then, we will delete your comments)_.

---

Reviewer1

MagicalSparkleReviewer

Recommendation (select one)

• [ ] Accept
• [X] Accept if (see below)
• [ ] Reject

Summary (1 para)

Contributors to GitHub projects may originate from many different time zones. How does time zone dispersion (or concentration) affect the distribution of work activity in these projects? The authors explore the relationship between timezone dispersion and work activity dispersion.They find that, while TOD work activity dispersion is associated strongly with time-zone dispersion, it is equally affected by the size of the project team.

Advocacy (accept since, reject since, 1 para)

The authors have conducted an interesting, well-considered study on time-zone dispersion in open-source projects. The study is clearly explained, seems to have been conducted reasonable, and the authors detail the possible threats to validity. This seems to be the first study to use circular statistics to study this problem, which adds novelty and corrects an oversight made by previous researchers. I am largely on the side of "accept" for these reasons.

However, I think this paper is hurt by the short format. There is very little space given to motivation, leaving me to spend much of my time thinking, “Why does this matter?” Further, there is very little discussion of the results. What are the implications of the findings? If I were a software manager, how could I use this data to assist in decision making? What are the actionable takeaways from this work? I think that the study is interesting enough that it could lead to an excellent full paper. With only four pages, I feel like I’m only reading half of the full story.

List of "Pros"

• Well-designed experiment.
• Good explanation of threats to validity.
• Use of circular statistics.

List of "Cons"

• Lack of discussion of the results.
• Motivation (why is this useful?) not very well explained.

Changes needed before I can recommend accept (if any)

• I would like to see expanded discussion. I think this would be a far more interesting full paper than a short paper.
• One other area for discussion - do these results correlate to commercial development? For instance, the authors saw low timezone variance, but high hour variance. With most commercial projects, my instinct would be to expect low variance for both (if most work is done in one location). It would be interesting to see how these results might differ between OSS and commercial projects. Even if the authors lack commercial project data, some speculation might be interesting.

[timm]

_AUTHORS: Important. Do NOT reply till all three reviews are here (until then, we will delete your comments)_.

---

Reviewer2

swan17Reviewer

Recommendation (select one)

[ ] Accept [X ] Accept if (see below) [ ] Reject

Summary (1 para)

Timezone dispersion in commercial projects has been considered an advantage as it can lead to round-the-clock activities (“follow the sun” work practice). The paper addresses whether this assumption holds for OSS projects on GITHUB. The main conclusion is that while time-of-day work activity is associated strongly with time-zone dispersion, it is equally affected (if not more strongly) affected by size of the project team.

Advocacy (accept since, reject since, 1 para)

The study encompasses a large sample of GITHUB large projects. It has been well conducted and explained and shows nicely the use of statistical analysis and threats to validity. All in all, the paper addresses its objective and the conclusion is nicely supported. However, the paper lacks a “motivation”. Why is important at all to tackle the question on hand? The authors simply state that they focus on OSS rather than on purely commercial projects, but they don’t motivate this at all.

List of "Pros"

• Interesting approach (“we are the first to study GitHub team work cycles using circular statistics”).
• Well-designed experiment

  List of "Cons"

• Criteria for project selection are not very clear to me (how these projects have been selected?)
• Main drawback is the lack of motivation
• How the conclusions relate to the assumptions in commercial projects?

Changes needed before I can recommend accept (if any)

• Add a few sentences wrt motivation
• Better explain the criteria selection of the projects sample

[timm]

_AUTHORS: Important. Do NOT reply till all three reviews are here (until then, we will delete your comments)_.

---

Reviewer3

swan17-bestreviewer

Recommendation (select one)

• [X] Accept
• [ ] Accept if (see below)
• [ ] Reject

Summary (1 para)

The paper describes an analysis of a set of open-source projects on GitHub w.r.t. the temporal dispersion of the committers and commits. It shows that the location of the committers has less effect on the time dispersion than the size of the project.

Advocacy (accept since, reject since, 1 para)

I think this is a very nice, compact paper on a very clear issue. The used analysis method seems very fitting. The results are discussed appropriately. I can see a lot of potential for a follow-up full paper with more discussions and maybe comparisons to closed-source software projects.

List of "Pros"

• Good, clear research question
• Good analysis
• Clear results

List of "Cons"

• Dense writing but still understandable
• Discussions could be longer to discuss the causes and effects

Changes needed before I can recommend accept (if any)

[MagicalSparkleReviewer]

Edited @timm's comment with my review. If I was supposed to repost, let me know and I can do that instead.

[swan2017xyz]

We thank the reviewers for their feedback, and suggestions to improve this paper.

@swan17Reviewer:

As requested, we will expand the motivation of this work, and the discussion of our results. We found after the submission that rendering the PDF with a newer Latex version gives us additional space that we can use for this purpose.

@MagicalSparkleReviewer:

As noted above, we will be able to expand the discussion, however we do not have data to discuss whether our results correlate to commercial development. We agree that such comparison would be very interesting. Unfortunately, including it in this short paper may not be feasible given time and space constraints.