search

greenelab / meta-review

Manuscript describing open collaborative writing with Manubot
https://greenelab.github.io/meta-review
Other
48 stars 21 forks source link

Initial submission to eLife: timeline & updates #8

Closed agitter closed 5 years ago

agitter commented 6 years ago

We are fairly close to having a full draft ready for submission. @dhimmel and @cgreene do you have any thoughts on a timeline? The remaining tasks are the Manubot section, addressing the many inline TODOs, and perhaps a figure to illustrate the issue and pull request system.

dhimmel commented 6 years ago

@agitter I'll get to work! Hopefully I can knock most of the Manubot content out next week... I begun yesterday!

agitter commented 6 years ago

Great! I think I nagged @cgreene enough to get it on his TODO list also.

Any thoughts on a figure? I have no graphic design skill so if I draft it, it would have to be screenshot-based.

cgreene commented 6 years ago

I have addressed the major TODOs that I think were assigned to me. :smile: I agree that a quick timeline is good.

agitter commented 6 years ago

What do both of you think about a figure? If we skip that, we could finish a full draft within a week.

dhimmel commented 6 years ago

What do both of you think about a figure? If we skip that, we could finish a full draft within a week.

I think we should have a figure or two. Here are areas of interest:

  1. screenshots of the contribution life cycle. Pull request / review comments > merged commit > Travis CI build / updated manuscript.
  2. citation exposé showing how the Manubot does citations / references
  3. Deep Review contribution plot. Here are possibilities: Rephetio, Cognoma, or a Gource-movie like this.
agitter commented 6 years ago

Those are all excellent ideas. 1 is the most essential because it will help readers understand the main idea and contribution.

For 3, are those figures easy to generate automatically from a commit history or does it take some manual effort? What would the cognoma graphic look like for us because we only have one repo? Markdown files by users? I like the static figures better than the video.

agitter commented 6 years ago

I'm following up on this figure idea

screenshots of the contribution life cycle. Pull request / review comments > merged commit > Travis CI build / updated manuscript.

I looked through some of the deep review pull requests. greenelab/deep-review#286 and greenelab/deep-review#347 stood out as two where there was discussion, user tagging, issue referencing that will illustrate advantages of GitHub. We could show panels with:

  1. Review comments image
  2. Updates in response to review image
  3. The final changes from the pull request image

@cgreene @dhimmel Any other ideas for what we want to show in this figure? If the changes were smaller in scope, it would be easier to show the edits to the full manuscript. The third image is probably too big to include. But the smaller pull requests didn't seem to generate as much review discussion of course.

cgreene commented 6 years ago

Hmm, I think I might just do part 1 and 2. It's nice to see the changes overall but the "before" part of that screenshot isn't too interesting. I agree though, it's a lot of text. I wonder if a schematic in the paper and the full example as a supplementary figure is better...

agitter commented 6 years ago

You're right, there isn't much interesting content per pixel there. I agree that a schematic would be much better than screenshots. I just don't have the graphical skills to make a nice one.

If none of us can make a schematic, should we create a new pull request designed to showcase these features instead of trying to find an existing one? It doesn't have to be from deep review, we could engineer a small change here to fix a TODO or add a reference.

cgreene commented 6 years ago

manubot-draft

I tried just to keep the focus on the most important bits. I made it in powerpoint, so I have a pptx file.

cgreene commented 6 years ago

here's the file in case you want to edit https://www.dropbox.com/s/7nqn77igjophuhz/manubot-draft.pptx?dl=0

agitter commented 6 years ago

This simplified version is much better than a raw screenshot. Want to make a pull request to add this figure? You can also put the pptx in content/images in case we need to edit it later.

I had de-prioritized this manuscript when I got caught up with others, but do we want to try to have a finished draft before we resubmit deep review? It would help simplify our deep review methods section.

dhimmel commented 6 years ago

do we want to try to have a finished draft before we resubmit deep review?

I actually think the Meta Review will be best after the deep review resubmission, when the contribution figure stabilizes. Both the Manubot and Deep Review are still progressing rapidly, so the Meta Review will get incomplete / outdated quickly.

It would help simplify our deep review methods section.

I'll take another look at the Deep Review methods.

agitter commented 6 years ago

when the contribution figure stabilizes

That makes sense now that we have an ongoing burst of deep review contributions.

agitter commented 6 years ago

Now that deep review has been resubmitted, I wanted to ping both of you (@cgreene @dhimmel) to think about our timing again. This is not really a good time for me to work on this, and I see you are both working on SciHub (among other things). However, the anticipated deep review coverage might provide a golden opportunity to submit meta review while interest in the process is high.

What do you think?

cgreene commented 6 years ago

I think that we will never have a "good" time to work on something like this, but this is probably the right time to work on it. We should press onward and get it submitted. I can shift some focus to this on or about Feb 5 if that's ok. I've got a trip next week that should be more or less all consuming.

agitter commented 6 years ago

Sounds good to me. I'll block off some time for this starting Feb 3 or 4.

@dhimmel I won't edit anything until we merge #26.

dhimmel commented 6 years ago

I won't edit anything until we merge #26.

Okay I'll try to resume work on #26 next week.

the anticipated deep review coverage might provide a golden opportunity to submit meta review while interest in the process is high

I agree. I think finishing v1 of the meta review shortly after the deep review is published makes sense.

agitter commented 5 years ago

@vsmalladi @slochower we should be ready to submit in the next few days. We initially plan to submit this to eLife as a feature article.

vsmalladi commented 5 years ago

@agitter Is there a final review that we should be looking out for that we will be approving?

dhimmel commented 5 years ago

Is there a final review that we should be looking out for that we will be approving?

The current version is the planned submission version, as far as I know. So if you have any further comments on https://greenelab.github.io/meta-review/v/b8eeea542ce238bbcaf2023add2aecb86ef726bd/, let us know!

slochower commented 5 years ago

I just added a PR with one grammar correction, one spelling change, and one sentence rearrangement.

I also made a comment on https://github.com/greenelab/meta-review/commit/17b42d887906c1ba81ce81fcc64e4a4319dc384d about a phrase I didn't understand.

I think it reads quite nicely. I started to get a sense that the introduction stressed review articles too heavily, but since the discussion mentions all the other types of Manubot documents out there, I think this forms a powerful sandwich. I think this is good to go from my point of view.

agitter commented 5 years ago

I submitted to the journal and tagged this version v1.0. I'll forward any email correspondence I receive.

agitter commented 5 years ago

I'm closing this. I don't see any remaining actionable items.