Best approach to providing revisions/feedback on methods documents

[houghb]

I don't necessarily have a clear answer to propose for this issue, but it seems like we should discuss it, maybe on the next phone meeting. In February we made a transition to beta testers interacting primarily through github instead of all the mediums we were using before (slack, github, e-mail, google docs, phone calls, etc). I wholeheartedly agree we had too many communication channels before, but the result of this change seems to be a lot less engagement from all parties in this process.

I understand the desire to use github, and I think raising/discussing specific issues is a great use case where it fits (assuming beta testers get engaged and look at issues when they have activity), but it doesn't seem like github is a good platform for reviewing methods documents or proposing changes to them.

For example, if I want to discuss or propose a change to a specific part of a methods document I need to create a new branch in this repository, edit the document in that branch with my changes, submit the new branch as a pull request, and then hope that someone sees it and comments on it. This is cumbersome, takes a lot of time, requires familiarity with git, and in the case of the one pull request I've done for this purpose it has not received any feedback for a couple weeks.

Google docs are much better suited for this purpose. One approach that might work would be to have a google doc for each method spec (for example, daily methods data cleaning, daily methods analysis, etc), and on the github README for that spec we could just have a link to the google doc (the README will only contain this link, not the methods). Once the spec is "finalized" then it could be ported to the github README. Finalized in this case would mean at the official release of v1.1 - after that time any future revisions would happen on github. Obviously this isn't ideal, but having to create a branch and edit the entire file and then submit a pull request for any suggested change to the methods seems unlikely to encourage participation.

[mcgeeyoung]

Thinking very much along the same lines...

[matthewgee]

@houghb @mcgeeyoung Thinking about how we can learn from what did, and did not, happen during last year's beta tests and how we make sure things happen differently, we tried the approach of having commentable methods google documents last year for each part of the specification that were in the agendas, with specific comment periods, and reminders in the calls about commenting but no commenting happened. I think the lack of comments has more to say about last year's process than the medium, but are their other issues around, as Blake has noted, the lack of engagement in the process, that the medium (github + google docs instead of just github) aren't going to fix that might be important to discuss and resolve here?

[houghb]

Thanks, @matthewgee, one thing about our approach from last year that was confusing for me was having the same document in so many places (we tried to have specs as github README's, but also tried to keep the same spec synchronized across several google docs so other stakeholders could comment on it, etc). I like the idea of having a single version of each spec in development, so that we always know which one is up to date and where to comment on it. I think google docs works better for this than anything github has to offer, which is why I suggested having an empty readme on github that just links to the google doc where the action happens during development.

In terms of other issues, I think we'll need to hear from other beta testers to get a sense of that. I've looked at who is "watching" this github repo, and who has joined the "repos" channel on slack, and most of the beta testers are not on those lists, which I think means that they are not being notified when changes happen to the repo or new issues are posted...

[mcgeeyoung]

@houghb Yes. That is what we are working on implementing.

[matthewgee]

We've now got this implemented as commentable google docs for pre-markdown methods revisions.

The daily methods google doc is found here. Closing this issue.