Author order - feedback requested

[agitter]

I'd like to openly discuss how we are going to order all authors in the final manuscript. Soon I will open a separate issue to discuss who will be listed as an author, what authors need to do to review and approve the manuscript, and what information we need from each author.

It will be too difficult to determine a total ordering of all ~20 authors by their degree of contribution. However, I would like to still acknowledge that some authors made very extensive contributions. I propose that each author be mapped into one of the following categories:

1. Organized and drafted at least one sub-section plus substantial additional contributions (e.g., reviewing pull requests, drafting multiple sub-sections, initial organization of ideas and issues)
2. Organized and drafted one sub-section
3. Contributed text and ideas but less than one full sub-section
4. Corresponding authors (@cgreene and me)

@cgreene and I would initially propose a mapping from all contributors to these categories, but then everyone would be free to discuss their contributions and placement. Once we finalize the mapping, authors would be randomly ordered within each category. If new authors are added later, they would be randomly inserted into the previously ordered list for the category. The category-specific lists would be sorted in the order listed above.

Please suggest improvements to this process.

[cgreene]

Tagging @dhimmel @gailrosen @brettbj @AnneCarpenter @traversc @bdo311 @enricoferrero @gwaygenomics @evancofer @sw1 @agapow @swamidass @XieConnect @jacklanchantin @j3xugit @DaveDeCaprio @AvantiShri to make sure it is seen by everyone on the contributors page + with an open PR. Due to the open PR - @AvantiShri should let us know if @akundaje contributed to the text from the PR.

[akundaje]

I did not contribute to the actual text but gave comments and suggestions on what should be included.

Anshul

On May 1, 2017 8:01 AM, "Casey Greene" notifications@github.com wrote:

Tagging @dhimmel https://github.com/dhimmel @gailrosen https://github.com/gailrosen @brettbj https://github.com/brettbj @AnneCarpenter https://github.com/AnneCarpenter @traversc https://github.com/traversc @bdo311 https://github.com/bdo311 @enricoferrero https://github.com/enricoferrero @gwaygenomics https://github.com/gwaygenomics @evancofer https://github.com/evancofer @sw1 https://github.com/sw1 @agapow https://github.com/agapow @swamidass https://github.com/swamidass @XieConnect https://github.com/XieConnect @jacklanchantin https://github.com/jacklanchantin @j3xugit https://github.com/j3xugit @DaveDeCaprio https://github.com/DaveDeCaprio @AvantiShri https://github.com/AvantiShri to make sure it is seen by everyone on the contributors page + with an open PR. Due to the open PR - @AvantiShri https://github.com/AvantiShri should let us know if @akundaje https://github.com/akundaje contributed to the text from the PR.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/greenelab/deep-review/issues/369#issuecomment-298349266, or mute the thread https://github.com/notifications/unsubscribe-auth/AAI7EYh4Wm75KbC3WSCgRShtBxmQLJmOks5r1fPVgaJpZM4NNEa_ .

[cgreene]

@akundaje : The readme says: "Anyone whose contributions meet the ICJME standards of authorship will be included as an author on the manuscript." Can you and @AvantiShri take a look at the standards http://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html#two and let us know.

[AvantiShri]

@cgreene I think Anshul's contribution was enough to be included as an author. Thanks.

[dhimmel]

I like the category idea. I'm not sure random ordering within category makes sense. Perhaps we could do better with a total word-by-word blame stat. However, if you do random, I'd suggest adopting a deterministic procedure with a predetermined random seed. I have a decent idea how to do this, but cannot disclose it a priori without subjecting it to gaming.

[agitter]

@dhimmel I agree the random ordering isn't perfect. I (or someone else) would use a random seed if we do go that route, but I haven't fully thought through the implementation. I prefer random to an alphabetical ordering within categories.

My hesitation with word-by-word blame is that it doesn't capture all types of contributions or initial drafts that laid out good ideas but were later edited. For instance, @gwaygenomics has been active in outlining the paper and its goals and adding detailed reviews in the issues. That isn't captured in the commit log. Likewise, you have done a ton of work on the build process and references, and it is hard to compare those lines of code to contributed text.

We could make more fine-grained categories. That diminishes the impact of the random ordering but increases our burden in mapping contributors to categories.

[agitter]

@cgreene I think Anshul's contribution was enough to be included as an author. Thanks.

Great! Keep in mind that we'll need all authors to approve the full manuscript, so @akundaje and @AvantiShri (and everyone else) can mentally budget some time this week or next to read the full paper and make local edits as needed.

[enricoferrero]

Sounds great, I think the 4 categories make sense. Within each category, while not perfect, I think ordering by contribution level would still be better than random or alphabetical order.

[DaveDeCaprio]

I also agree with the categories. I would be comfortable with a random ordering, as I could see it being difficult to try to come up with something truly equitable otherwise.

[agitter]

@enricoferrero Do you have an idea for a metric that would work to sort by contribution within each category? I haven't been able to think of anything suitable yet.

[j3xugit]

Any order is ok with me.

Anthony Gitter notifications@github.com于2017年5月1日 周一下午12:42写道：

@enricoferrero https://github.com/enricoferrero Do you have an idea for a metric that would work to sort by contribution within each category? I haven't been able to think of anything suitable yet.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/greenelab/deep-review/issues/369#issuecomment-298383973, or mute the thread https://github.com/notifications/unsubscribe-auth/AKR63rLDX9xpEoLpgyE-fpsc6rn0-dZXks5r1hl7gaJpZM4NNEa_ .

--

---

Professor Toyota Technological Institute at Chicago 6045 S. Kenwood Ave. Chicago, IL 60637 fax: 773 834 2557, Google Voice: 773 359 3721 http://ttic.uchicago.edu/~jinbo/

[enricoferrero]

@agitter I was just thinking number of lines added/removed or number of merged pull requests maybe? Obviously not all contributions translate to commits and pull requests but they might be a good enough proxy.

P.S.: Random ordering works perfectly well for me.

[gailrosen]

I think it should be the order of the contributions (which may be more subjective than lines added/removed) but should be determined by Greene and students perceived contributional content.

On Mon, May 1, 2017 at 2:57 PM, Enrico Ferrero notifications@github.com wrote:

@agitter https://github.com/agitter I was just thinking number of lines added/removed or number of merged pull requests maybe? Obviously not all contributions translate to commits and pull requests but they might be a good enough proxy . P.S.: Random ordering works perfectly well for me.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/greenelab/deep-review/issues/369#issuecomment-298402534, or mute the thread https://github.com/notifications/unsubscribe-auth/AE-Yub7jYWlucqclP8iJ6_AbxptOw67cks5r1isUgaJpZM4NNEa_ .

-- Gail L. Rosen, Associate Professor Electrical and Computer Engineering Director of the Center for Biological Discovery from Big Data http://biomed.drexel.edu/labs/biodiscovery/ Drexel University Webpage/Contact info: http://www.ece.drexel.edu/gailr

[agapow]

I'm fully happy with whatever decision @agitter and @cgreene come up with. Obviously it's an awkward difficult decision to which there may not be any good solution. Propose one and roll with it.

[agitter]

@gailrosen the distributed nature of the review has made it hard in practice for us to rank these contributions subjectively. For example, I reviewed and approved most (but not all) of the Study sub-section, so perhaps I could order those. But @cgreene reviewed and merged most of Categorize, so we would have a hard time combining ranked contributors across those sections. I'm open to more suggestions though.

@enricoferrero I was thinking about these metrics as well, but I've found cases where they break down. Some pull requests reflow large blocks of text, which shows up as many lines added and removed. Others are important edits but only change a few sentences, which shouldn't count as much as a new section. Total commits are also problematic because initially we merged individual commits but later we started squashing and merging.

[dhimmel]

I was thinking about these metrics as well, but I've found cases where they break down. Some pull requests reflow large blocks of text, which shows up as many lines added and removed. Others are important edits but only change a few sentences, which shouldn't count as much as a new section.

@agitter, I was thinking of a number of words by author metric. This would sidestep many of the issues you mention with diff-based metrics. I looked a bit and I think there are potentially methods to compute word count: e.g. see https://github.com/d33tah/wordblame/issues/1. I agree word count shouldn't be used as the only metric for subranking. If you wanted to be devilish, you could weight the random ordering by word count.

[AnneCarpenter]

But wait, I use very long words! Just kidding, it's nice that everyone is putting so much thought to this. I deeply suspect most authors don't really give any care at all to the precise ordering (at least I'm in this category!)

[evancofer]

My preference is alphabetical within category. I am also fine with random within category, but it might require some additional work to do so in a transparent manner.

[cgreene]

I am really bad at this thing. I missed @blengerich and @alxndrkalinin! I'm sorry guys - you are definitely in (assuming you approve, etc). We're happy to have you and this was my oversight as I scrolled through the list and tagged people!

[cgreene]

If anyone else notices someone missing, PLEASE let me know.

[blengerich]

No worries @cgreene. I'm happy with any ordering.

[nafizh]

Is the contribution window still open or the manuscript is done?

[cgreene]

@nafizh : We are at the stage of wrapping this up. The manuscript is not yet done. We need to finish by May 14 so that we can get it to the journal by the agreed upon date (May 15). If there are key missing pieces or concepts that you think should be added, feel free to file a PR to address them. We're at the stage now where entire new sections aren't something that we can review, merge, and revise. Any changes would need to be tuning of what's already here.

[agitter]

@dhimmel I thought about a weighted random ordering, but then we have to decide an exchange rate for things like words contributed versus time spent reviewing pull requests. My intention to have sufficiently fine-grained contribution classes such that authors within a class have roughly equal contributions.

@evancofer I agree that a random ordering would need to be transparent. We could see the random seed here in an arbitrary way before running the ordering code. That way whoever runs the code (likely me) can't rerun it with different seeds until a particular outcome comes out.

I added a proof of concept author ordering notebook on my fork.

[cgreene]

In preparation for this, I've created a contact and approval form and updated the README. Please file the requested PR to mark your approval of the manuscript and to provide the information that we'll need to submit it.

[cgreene]

Tagging @yfpeng @davharris since I think they have contributed since the list was compiled.

[davharris]

I'm happy with any ordering. I just added myself to the bottom

[agitter]

@jisraeli also submitted a pull request after our initial list was compiled.

[yfpeng]

My supervisor, Dr Zhiyong Lu, help me compose the text. We are happy with any ordering. I just added both of us to the bottom.

[gailrosen]

Hi all,

I have not needed an ORCID, but I know that some journals may now require it. Should I go ahead and get an ORCID?

best, Gail

On Thu, May 11, 2017 at 7:46 PM, Yifan Peng notifications@github.com wrote:

My supervisor, Dr Zhiyong Lu, help me compose the text. We are happy with any ordering. I just added both of us to the bottom.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/greenelab/deep-review/issues/369#issuecomment-300946087, or mute the thread https://github.com/notifications/unsubscribe-auth/AE-YuaClGzQe-30n1u4mCGRxyDug2ukpks5r453HgaJpZM4NNEa_ .

-- Gail L. Rosen, Associate Professor Electrical and Computer Engineering Director of the Center for Biological Discovery from Big Data http://biomed.drexel.edu/labs/biodiscovery/ Drexel University Webpage/Contact info: http://www.ece.drexel.edu/gailr

[cgreene]

@gailrosen : I do not know for sure that it is required, but since it's something that you'll probably need to do soon anyway, it may be helpful. Some journal submission systems do a nice job of using these.

[agapow]

I believe that Roy Soc requires an ORCID only for the submitting / primary authors, not for all (although it does recommend them).

[dhimmel]

@gailrosen I highly recommend getting an ORCID and using wherever you can! Otherwise, it's really difficult to uniquely identify a scientist. Here's an analogy: ORCID is to researcher as DOI is publication!

As a project that relies heavily on metadata for referencing, I think we should strive to have all authors supply their ORCIDs!

[agitter]

@dhimmel you'll be happy to know that @gailrosen did go ahead and get an ORCID (#454).

I agree that we can ask all authors to provide one.

[agitter]

The random number generator seed I will use for author ordering will be generated in a pre-defined way that I cannot directly control. I will merge #470 and create a new commit with a squash merge. The commit id of that new commit will be used as the seed for the author ordering.

[agitter]

Per 06edef4bee392038b3e0403b0fa4208e11ddcc5d our random seed is 0x06edef4bee392038b3e0403b0fa4208e11ddcc5d (in hex)

[cgreene]

Hi thread! We have another issue. I need you to create accounts at JRS Interface, as I can't associate your ORCIDs with the paper without this.

side note: @gailrosen and @j3xugit - I was adding you manually so you may have gotten an email already about this - please log in and add your ORCIDs.

Everyone else, please create an account here + add your ORCID: https://mc.manuscriptcentral.com/jrsi

Then post in this thread that you have done so and I'll get you added. Thanks!

[yfpeng]

Done. Thank you!

[cgreene]

Can confirm - after @yfpeng creates an account things work. Ok! I will :+1: your comment after you post here that you've created an account, and I add you to the manuscript.

[dhimmel]

Account created with username dhimmel.

[gwaybio]

Account created with username gregway@mail.med.upenn.edu

[agitter]

Username agitter

[j3xugit]

Looks like that I have not received an email from JRSI.

On Mon, May 22, 2017 at 2:04 PM, Casey Greene notifications@github.com wrote:

Hi thread! We have another issue. I need you to create accounts at JRS Interface, as I can't associate your ORCIDs with the paper without this.

side note: @gailrosen https://github.com/gailrosen and @j3xugit https://github.com/j3xugit - I was adding you manually so you may have gotten an email already about this - please log in and add your ORCIDs.

Everyone else, please create an account here + add your ORCID: https://mc.manuscriptcentral.com/jrsi

Then post in this thread that you have done so and I'll get you added. Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/greenelab/deep-review/issues/369#issuecomment-303191214, or mute the thread https://github.com/notifications/unsubscribe-auth/AKR63nt4pLrmYJd8FZVAaz7XypcvVzcwks5r8dxBgaJpZM4NNEa_ .

--

---

Professor Toyota Technological Institute at Chicago 6045 S. Kenwood Ave. Chicago, IL 60637 fax: 773 834 2557, Google Voice: 773 359 3721 http://ttic.uchicago.edu/~jinbo/

[gailrosen]

I did not either, but I just created an account under gailr@ece.drexel.edu

On Mon, May 22, 2017 at 3:57 PM, j3xugit notifications@github.com wrote:

Looks like that I have not received an email from JRSI.

On Mon, May 22, 2017 at 2:04 PM, Casey Greene notifications@github.com wrote:

Hi thread! We have another issue. I need you to create accounts at JRS Interface, as I can't associate your ORCIDs with the paper without this.

side note: @gailrosen https://github.com/gailrosen and @j3xugit https://github.com/j3xugit - I was adding you manually so you may have gotten an email already about this - please log in and add your ORCIDs.

Everyone else, please create an account here + add your ORCID: https://mc.manuscriptcentral.com/jrsi

Then post in this thread that you have done so and I'll get you added. Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/greenelab/deep-review/issues/369# issuecomment-303191214, or mute the thread https://github.com/notifications/unsubscribe-auth/ AKR63nt4pLrmYJd8FZVAaz7XypcvVzcwks5r8dxBgaJpZM4NNEa_ .

--

---

Professor Toyota Technological Institute at Chicago 6045 S. Kenwood Ave. Chicago, IL 60637 fax: 773 834 2557 <(773)%20834-2557>, Google Voice: 773 359 3721 <(773)%20359-3721> http://ttic.uchicago.edu/~jinbo/

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/greenelab/deep-review/issues/369#issuecomment-303203452, or mute the thread https://github.com/notifications/unsubscribe-auth/AE-YuagYnGSJxgTxi6STTAXVb5-8A6DBks5r8eiVgaJpZM4NNEa_ .

-- Gail L. Rosen, Associate Professor Electrical and Computer Engineering Director of the Center for Biological Discovery from Big Data http://biomed.drexel.edu/labs/biodiscovery/ Drexel University Webpage/Contact info: http://www.ece.drexel.edu/gailr

[AnneCarpenter]

Done! username anne@broadinstitute.org

[bdo311]

also done! brian_do@hms.harvard.edu

On Mon, May 22, 2017 at 4:15 PM, Anne Carpenter notifications@github.com wrote:

Done! username anne@broadinstitute.org

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_greenelab_deep-2Dreview_issues_369-23issuecomment-2D303208037&d=DwMCaQ&c=WO-RGvefibhHBZq3fL85hQ&r=e1hzu-U8qikZfP5tgHbADYs9i6YxmVWfXPTzmp1c7ds&m=ZWZSaQqVja0CgWOcx790AX6XQQJHj-KQSVyw6JAtvp8&s=YGxsQU_w8Qk2Mbj1ZLmVhKj_S1xkFa3z0efMcFApbc4&e=, or mute the thread https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AIIbiKhSo9PrvZV-2DaP0w7H1vzAePFjZ6ks5r8ezjgaJpZM4NNEa-5F&d=DwMCaQ&c=WO-RGvefibhHBZq3fL85hQ&r=e1hzu-U8qikZfP5tgHbADYs9i6YxmVWfXPTzmp1c7ds&m=ZWZSaQqVja0CgWOcx790AX6XQQJHj-KQSVyw6JAtvp8&s=DgPo5CyZ_5usIPafBb7Euf7P3El7TQQgdP_dtbSmiP4&e= .

[blengerich]

Done! blengeri@cs.cmu.edu

[jacklanchantin]

Created under jacklanchantin@gmail.com

On Mon, May 22, 2017 at 4:22 PM, Ben Lengerich notifications@github.com wrote:

Done! blengeri@cs.cmu.edu

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/greenelab/deep-review/issues/369#issuecomment-303209677, or mute the thread https://github.com/notifications/unsubscribe-auth/AFJb_AT46yR3WHxsIdPqUaGZcQBTs8WUks5r8e6LgaJpZM4NNEa_ .

[evancofer]

I have created said account. The associated email is "ecofer@princeton.edu" (without quotes).

[XieConnect]

I just created an account under: wei.xie@vanderbilt.edu Thanks.