search

tqdm / tqdm

:zap: A Fast, Extensible Progress Bar for Python and CLI
https://tqdm.github.io
Other
28.32k stars 1.35k forks source link

update paper #887

Open casperdcl opened 4 years ago

casperdcl commented 4 years ago

Discuss updating paper (archived at https://doi.org/10.21105/joss.01277) as per https://github.com/openjournals/joss-reviews/issues/1277.

casperdcl commented 4 years ago

In an attempt to avoid any debate (so we can focus on tackling other open issues), hopefully we can get an immediate decision:

Reading the editor-in-chief's comments very carefully, in particular where and when others are @mentioned, the impression I get is he has done some research and may be implying authors: @casperdcl, @lrq3000, tqdm developers; acknowledgements: @noamraph. I might be wrong and it would be great if @arfon could confirm either way. Further I would suggest (as per @hadim's request) to also add @hadim in the acknowledgements. I would also suggest adding @kmike if he is amenable. I am unsure about adding others but will not refuse any suggestions.

To further prompt a decision:

  1. I reiterate my willingness to accept the editorial team's decisions.
  2. I think one of the main responsibilities of a journal is to (with the aid of/via the reviewers and editorial team) define and make decisions on authorship.

I understand that is a big request with a lot of responsibility and research associated, especially considering the voluntary nature of the JOSS team's work. However I would very much appreciate if at least a suggestion if not a decision is provided.

noamraph commented 4 years ago

Hi! Here I am!

Arfon wrote: "all significant contributors should be included in the author list." I consider myself to be a significant contributor, by writing the initial working version. I acknowledge that a lot of significant work has been done since, probably with much more time spent than my initial effort. However, I still consider myself to be a significant contributor.

Cheers, Noam

casperdcl commented 4 years ago

Hey @noamraph,

Happy to hear from you.

Unfortunately though in this instance I disagree for a long list of reasons. However I really hope to avoid or at least shorten any debate (whatever the outcome) if we could get a response from @arfon here...

arfon commented 4 years ago

@casperdcl, @lrq3000, tqdm developers; acknowledgements: @noamraph. I might be wrong and it would be great if @arfon could confirm either way.

Apologies if this is how my comment was interpreted, this is not what I was trying to imply because...

I think one of the main responsibilities of a journal is to (with the aid of/via the reviewers and editorial team) define and make decisions on authorship.

I disagree. The journal's responsibility is to ensure that there has been ethical behaviour on the part of authors, reviewers, and editors. It it not the responsibility of the journal to decide who should be an author on a given paper. As I said in this comment authorship is a challenging subject and something that the authors/contributors of the package are best-placed to decide between themselves.

To be absolutely clear here I think if @noamraph wants to be an author, he should be.

Hopefully this is useful additional context. I am hoping this additional input from myself will be sufficient to guide you all in your decision. As I said over in https://github.com/openjournals/joss-reviews/issues/1277#issuecomment-578402029, this is a conversation that needs to happen between you all, and really isn't something that I can/or should be steering.

arfon commented 4 years ago

Hi all, I'd like to introduce @gkthiruvathukal who is a member of the JOSS editorial team. @gkthiruvathukal has kindly volunteered to act as a neutral mediator/third party in these discussions. As past Editor in Chief of CiSE he has a wealth of experience with discussions such as these.

gkthiruvathukal commented 4 years ago

@arfon Thanks for the introduction. I've taken the time to look at tqdm. It's great to see all of the work that has gone into this effort. As the neutral mediator (and observer in general) I'd like to encourage us to start from the git fame output shown on the landing page.

The git fame output shows that there are multiple contributors who probably should be recognized as co-authors, per JOSS guielines. I've done some preliminary analysis below.

Strong case for co-authorship: @casperdcl @lrq3000 @noamraph

Strong case for acknowledgments and possible co-authorship: @altendky (top 3 on git fame list, possible co-author?) @chengs @mjstevens777 @hadim @hmike

Again, I remind everyone that this is coming straight from the tqdm landing page.

In Software Engineering, I teach that not all contributions can be measured by LOC or number of commits. There are multiple ways to make an impact, and sometimes an enormous amount of impact can be made without the singular focus on LOC (or commits). For example, there can be developers who write good tests, configuration scripts, and documentation. These all make for great projects, and tqdm seems to have most of the good things in SE going for itself. This project is lucky to have such a great group of contributors, and it is my hope that you all can work to understand the importance of your software being well-cited as opposed to individuals.

@arfon and I have discussed this case, and we don't want to make decisions for you. We are both in agreement that this submission needs to recognize all co-authors in order for it to remain a JOSS submission, so if you are mentioned above, perhaps you can respond (concisely) with whether you agree with the above initial analysis. If you think you should be on a different list (co-authors or acknowledgments) than my initial triage, please make this clear in your follow up. I promise to hear everyone's input and help you to reach a conclusion that is good for the project and those who have made the most significant contributions to its success. This is a nice project and is the sort of contribution we really like to see submitted to JOSS.

I'd like to see if we can bring the authorship issues to a resolution by Friday, 31 January, so JOSS can decide on the next steps, which will involve revising the submission one way or another.

altendky commented 4 years ago

@gkthiruvathukal, I have no concerns with your initial groupings for attribution.

@lrq3000's comments about lack of attribution of changes had me a little worried as I remembered some funny feelings with some of my contributions. I think the primary missing part on my contributions was that two PRs showed as closed, not merged (#673, #622). The commits they were closed by do show me as author but for some reason have @casperdcl as committer. #674 (adding four words on one line...) was closed and a different solution committed without referencing me (no complaints here). #598 (my only significant contribution) was merged per normal GitHub methods. I closed #590 because I took a different approach in #598 that replaced it. I won't speak to @lrq300's issues but the only comment I would have related to mine is that it would probably be good practice to use the GitHub merge button so it shows up as merged in GitHub. It supports merge, squash-merge, and rebase so hopefully there's sufficient flexibility to satisfy personal git preferences.

Based on the infamously questionable metric of lines of code I would give @lrq3000 'author' level attribution. https://github.com/tqdm/tqdm/graphs/contributors (5220 lines compared to @casperdcl's 23,267 lines). It sounds like if I actually bothered to research it I would give 'author' level attribution for other reasons as well.

Based on being the original author (I think there isn't contention on this 'fact'?) I would give @noamraph 'author' level attribution.

Should I be an 'author'? Aside from the whole conflict of interest etc in me weighing in on credit due to me... I think it depends. Is an author someone that authored a block of code for tqdm? Or is an author someone that has contributed primary architecture or a large featureset? I certainly contributed one non-trivial bug-fix. I certainly did not design or author a significant portion of tqdm. I suspect that the sensible answer is to put me in the 'acknowledgements' group, not co-author. If I get co-author for my one bug-fix then we should look at everyone else that made non-trivial bug fixes and feature additions and include them as well.

Oh, to be complete, I'd put @casperdcl in the 'author' group as well. :] And no, I'm not going to bother to justify that.

hadim commented 4 years ago

I would give @noamraph, @lrq3000 and @casperdcl 'author' level.

As for me, I don't think I should choose for myself rather I will let others choose for me. That being said I am fine either way. The only thing I ask is to at least be cited in a 'history' paragraph.

My contribution to the code is much more modest than the devs I've cited above but I would like to highlight that I am the one who originally forked noamraph/tqdm repo to create tqdm/tqdm. Now to be honest I don't remember exactly everything so others please feel free to tell if I am wrong.

Quickly after the creation of tqdm/tqdm a bunch of devs showed up (including @lrq3000, @casperdcl and @kmike ) and showed a very strong interest in the project. Everyone started to open issues and create PRs and very quickly the project became very popular. I remember giving up quickly on the development after the fork and letting the other devs maintaining tqdm.

I don't know if the project would be different today without me forking it. I guess probably not, since someone would have done it anyway at some point.

That was my modest contribution to the debate. I hope everyone will get their deserved acknowledgment/authorship without any more drama :-)

gkthiruvathukal commented 4 years ago

@hadim and @altendky, thanks for your quick follow-up. I am going to continue observing, but my initial reaction is that you probably deserve more than an acknowledgment for your contributions as significant contributors. I always take an inclusive approach to contributions, no matter how large or small. Hopefully, the lead contributors will chime in with whether we could live with the top contributors as minor co-authors. We can easily arrange the co-authors in any order. I don't think there is much doubt about @casperdcl, @lrq3000, and @noamraph but am at this point hoping the three of them could agree to be co-authors and invite any additional authors with a strong case for inclusion.

altendky commented 4 years ago

@gkthiruvathukal, agreed as far as tending towards more inclusivity. Is there something that restricts to co-authors and acknowledgments as opposed to adding a contributors group in between? I wouldn't feel slighted being 'merely' acknowledged but if there's any contention on the topic for others then perhaps more categories would help (I mean... three anyways, not 17).

lrq3000 commented 4 years ago

Hello everyone! Thank you all (including Casper) so much for your feedbacks! Sorry for my delay, I tend to always spend more (too much?) time on research before doing anything :-) Thank you very very much @gkthiruvathukal and @arfon for accompanying us and your very valuable insights on this issue!

About the main issue at hand, here are my suggestions:

Questions that need to be resolved:

For efficiency, I have created an etherpad (collaborative text editor) with a rough outline of my idea of the changes we could make, so that everyone here can contribute (if there is any vandalism, I can make it private and invite manually), and it's still of course meant to change depending what we agree on:

https://board.net/p/tqdm-paper

(Please enter your name in the upper right corner, language can be changed in the settings if it's not autodetected from your browser)

lrq3000 commented 4 years ago

(For those who received the email notification, I have change the URL for the pad to be in English by default, please prefer to use this URL: https://board.net/p/tqdm-paper )

gkthiruvathukal commented 4 years ago

Responding to @lrq3000's questions:

Q: Arfon, you said the paper will be reviewed through a "fast-track", so @gkthiruvathukal do you know if this means we should stick with minimal modifications (ie, we shouldn't rewrite significant portions of the paper), right? Would a new History section (along with Acknowledgements) be ok?

A: Yes, the modifications would be minimal, and a complete rewrite should not be necessary.

Q: How do we organize the attributions? There are three possible places: In paper's authors list, In paper's body in context, In an Acknowledgements section.

A: Speaking as an observer, I'd like to see as many authors as possible. I know tqdm is important, but let's remember that the paper with the record number of authors on LHC (Large Hadron Collider, a super important project, too) is more than 5,000 authors, https://www.nature.com/news/physics-paper-sets-record-with-more-than-5-000-authors-1.17567. Did everyone contribute equally to this paper? Did everyone even write it? Or do the actual research? We'll never know, but they're all co-authors. We have a much smaller number of potential authors here? I'd like to see as many included as possible, provided their contributions can be justified in quantitative or qualitative terms.

Q: In relation to the previous point, we also need to assess the contributions to the code. There are several tools for that (next I'll experiment with them). @gkthiruvathukal is there a tool (apart from git fame) you would advise to explore in more fine grained details the git history? We should also review issues since some were not properly merged in.

A: I continue to research on this topic, but there are no really great tools, which is why I'm doing research on software metrics (hoping to change the state of affairs). One possible way is to look at other data that might show activity. You seem to have a fairly active issue tracker. Perhaps you can look at how many members of the team helped to raise and/or resolve issues. Did any contributors help, specifically, with user-facing aspects of the software (testing, docs, etc.?)

In the end, however, this can't be left entirely to data. It comes down to team dynamics and people advocating for others who may have made significant contributions. It seems like we already have convergence on some key contributors. The question is, are there any other minor contributors who are worthy of co-authorship? I would like those in a strong position to speak for those who may not be willing to advocate for themselves. Once I see all of the responses, I will synthesize a list of co-authors. Again, I must stress that Arfon and I do not want to make the decisions for you. And similar to @lrq3000, I prefer to focus on research. Oddly, this project is helping to motivate the need for metrics, which is the subject of some of my current research. (Lest I digress.)

I won't speak to the "end game" because @arfon and the board are still discussing how to handle papers where the DOI and its metadata may need to be changed. My goal is to help mediate the (ethics) issues that were raised and help everyone to bring the matter to a graceful (and civilized) conclusion. There seems to be a lot of goodwill here, and I am pleased with the responses I have seen thus far. I'm optimistic we can come together to get the co-author list right!

lrq3000 commented 4 years ago

@gkthiruvathukal Thank you so much for your again very enlightening reply! And I hope you'll get all the funding needed to pursue your goal, it would really help to have better metrics indeed ;-)

I get what you said, although I have I think a fairly good memory of what happened and all contributors (I tend to have a good memory of those things, as I am always grateful to anyone who freely spend time to help others such as in opensource projects), I don't know what happened after, so I'm now going to try to explore the history with tools and manually to get a more comprehensive view and then I'll offer suggestions :-) (this may take some time, any help is welcome, if anyone has some contributors in mind please feel free to cite them!)

gkthiruvathukal commented 4 years ago

Thanks for your support, @lrq3000. I had some funding for work on this topic, but it was only enough to prove some initial concepts. If interested to hear more, we can correspond sometime after getting tqdm's issues sorted out!

It would be great if we can get a HISTORY.md document produced. This would go a long way to clarifying how the project came to be--and how it evolved. Although I suggested Friday as a deadline for gathering initial input, that doesn't mean we can't wait, especially if it means we will include everyone who deserves to be included.

lrq3000 commented 4 years ago

@gkthiruvathukal Oh yes, I would be delighted to hear more!

Yes great idea for HISTORY.md, I'll direct my effort towards drafting that then :-)

lrq3000 commented 4 years ago

Actually, @casperdcl did a great job documenting changes and PRs in Releases, so maybe we can use that to build a HISTORY.md?

lrq3000 commented 4 years ago

Here's a first attempt HISTORY.md generated by the wonderful github-changelog-generator. It neatly shows the contributor for PR that were merged in. I think this should be usable as a basis to refine :-)

noamraph commented 4 years ago

Hi everyone, apart from my wish to be listed as an author, I would accept any decision that you take. I have no problem with the list of authors being inclusive. I'm grateful for all the hard work that you spent making my little project into something so awesome!

Regarding history, I first wrote tqdm where I worked, at about 2006. Iterators where pretty new, and I really liked the possibility of showing a progress meter with only adding a few characters. I chose the name "tqdm" because I looked for a short and unique name, and I really like Arabic. (I'm a Jewish Israeli). "tqdm", which is Arabic "تَقَدُّم" in Latin letters, pronounced "taqaddum", simply means "progress".

The small library turned out to be very popular in my work place, so I decided to open source it.

kmike commented 4 years ago

Hey! I think that any arrangement which includes @casperdcl, @noamraph and @lrq3000 as authors and mentions @hadim in some way is fine; their contributions were critical for having tqdm as it is now. It seems most proposals here suggest something similar anyways 👍

gkthiruvathukal commented 4 years ago

Ok, I am synthesizing here. As you read this, please keep in mind that I do not know any of you personally and am focusing on the input you've given and what git tells us.

I think there is strong consensus for @casperdcl, @lrq3000, @noamraph (probably in that order). Based on my earlier report, I think there is a strong case for @altendky as the 4th author, based on my earlier analysis and his follow up (where he took great care not to say whether he should be a co-author). In his case, however, he seems willing not to be a co-author but has made non-trivial contributions to the success of the project. For this reason alone, I take the view that he should be a co-author. I'm aware that number of commits and lines of code are not the only measures, but in looking at the actual commits, I am convinced his contributions are significant.

As I mentioned, I am hoping to have any and all input by Friday of this week, after which I think @arfon can take the next editorial steps. Based on the information I have in git and on this issue thread, I feel comfortable moving toward these four as co-authors in the order mentioned but would certainly appreciate if all who have commented would react (an emoji will do) to this proposal.

casperdcl commented 4 years ago

There are two very distinct issues: ongoing maintenance of the library (paramount importance), and secondly the publication joss.01277.

1) tqdm maintenance practices

[@altendky] I remembered some funny feelings with some of my contributions. I think the primary missing part [...]

This implies there are other more minor things you haven't mentioned. Please mention them! Also let me know if I haven't responded to any of your other points.

[@altendky] two PRs showed as closed, not merged (#673, #622). The commits they were closed by do show me as author but for some reason have @casperdcl as committer.

Did you check "allow commits from maintainers" when opening your PR? If not, or if I felt there was a lot more work I had to do to tidy up, I would've rebased your commits. This is extremely common good git practice. You will show up as the author (both on the commits and in any metrics such as those generated by GitHub, and those generated by git fame). I want you to show up as the author! You obviously won't be the committer on such rebased/cherry-picked commits unless I do something really clandestine contrary to the design of git. Author takes precedence over committer. Please just take it to mean "the committer's machine was incidentally used at some point."

[@altendky] #674 (adding four words on one line...) was closed and a different solution committed without referencing me (no complaints here).

Yes, it looks like I accidentally independently implemented something which made your PR redundant. Based on the discussion it was also unclear whether it should have been required at all. In any case, at the time, I acknowledged all of this explicitly and also thanked you and added you to the list in the README.

[@altendky] #598 (my only significant contribution) was merged per normal GitHub methods [...] the only comment I would have related to mine is that it would probably be good practice to use the GitHub merge button so it shows up as merged in GitHub

Sort of, I think because you kindly helped with rebasing, debugging and tidying everything so I didn't have to do much! It wasn't "normal" in the sense that I never use the "merge" button as merging locally and pushing is much more explicit and thus reliable. I try my best to ensure that whatever I do locally shows up correctly on GitHub but in general don't trust the merged-vs-closed/fixed-vs-closed status of PRs/issues, which is why I use a whole bunch of other metrics when acknowledging contributions.

[@altendky] Should I be an 'author' [...] suspect that the sensible answer is to put me in the 'acknowledgements' group

You are already an author (of tqdm). See my first comment. Regarding joss.01277, though, see the next section (basically happy for you to be acknowledged, and hopefully if JOSS updates their guidelines have you as author).

[@hadim] I hope everyone will get their deserved acknowledgment/authorship without any more drama.

Referring solely to the library (incl. docs/website but not the paper, which I'll address separately below) please let me know if you feel you need more prominent acknowledgement. Let's face it, the library's docs have been viewed over 818 thousand times; several orders of magnitude more than any paper could hope for.

[@gkthiruvathukal] You seem to have a fairly active issue tracker. Perhaps you can look at how many members of the team helped to raise and/or resolve issues.

As I mentioned above, I don't trust the "resolved" status. "Participated" might be a better thing to look for.

[@gkthiruvathukal] sometimes an enormous amount of impact can be made without the singular focus on LOC (or commits). For example, there can be developers who write good tests, configuration scripts, and documentation [...] Did any contributors help, specifically, with user-facing aspects of the software (testing, docs, etc.?)

[@gkthiruvathukal] This project is lucky to have such a great group of contributors

[@gkthiruvathukal] understand the importance of your software being well-cited

It's not important for FOSS software to be well-cited compared to the importance for it to be well-used.

[@gkthiruvathukal] we don't want to make decisions for you

Yes, I understand you don't want to (and that's within your rights), but the consequences are MUCH worse.

[@noamraph] Regarding history, I first wrote tqdm where I worked, at about 2006 [...] turned out to be very popular in my work place, so I decided to open source it.

I hope you had authorisation from your company https://www.theregister.co.uk/2019/12/12/nginx_moscow_office_raided/.

2) JOSS publication

Oh dear. This is exactly the sort of libellous argument ad-hominem which could have been avoided if JOSS could step in. I understand JOSS is volunteer-run, does great work, and is under no obligation to help, though it would be most useful if JOSS policies are revised to be more helpful.

I have mixed views about various posts in this issue. I am tempted to respond point-by-point, however I believe my views (and indeed anyone else's) are irrelevant to the underlying issue. Attempting to change people's views (in order to avoid the underlying issue) is not something I am comfortable with. Everyone is entitled to their beliefs. Also note that who meets the current and/or future JOSS definition of paper author will unlikely be the same as what anyone feels (i.e. "who do I wish were paper authors" versus "who actually are the paper authors").

To summarise the issue:

There are some considerations which hopefully will make JOSS better:

  1. Introduce explicit guidelines:
    • The IEEE definition of authorship and citation guidelines are a good place to start.
    • Modifications would of course be needed tailored to a FOSS-specific journal.
    • It is unreasonable to expect non-legally trained software devs to discuss this issue with no concrete guidance. It is also unethical to force volunteer FOSS devs to spend time debating this instead of just telling them what to do so they can move on to contributing more code and ideas.
    • Any numeric thresholds (e.g. min 5% LoC contribution), while bound to be wrong, are less wrong than not having thresholds at all (e.g. asking a teacher to mark and accepting marking errors will occur rather than asking students to rank themselves).
  2. Resolve discrepancies between current published guidelines and unpublished internal guidelines/beliefs:
    • Published guidelines https://joss.readthedocs.io/en/latest/review_criteria.html#authorship:

      • submitting author has made a 'substantial contribution' [...] determined by the commit history"

      • the full list of paper authors seems appropriate and complete

      • does not state "all 'substantial' project contributors should be paper authors," which I think would be a harmful requirement but nevertheless something JOSS is entitled to add.
    • Unofficial JOSS comment https://github.com/tqdm/tqdm/issues/887#issuecomment-578495632

      • [@arfon] It it not the responsibility of the journal to decide who should be an author on a given paper

      • Conflicts with the review criteria. Criteria for deciding are indeed given (but evidently need to be improved upon).

      • [@arfon] authors/contributors of the package are best-placed to decide between themselves

      • Once again seems to be confounding the issue by being ambiguous about whether we are talking about author (which I define to be of a paper) and contributor (which I define to be any activity on GitHub).
      • Also stating that authors should decide who are authors is a tautological logical fallacy.
    • Unofficial JOSS comment https://github.com/tqdm/tqdm/issues/887#issuecomment-578523686

      • [@gkthiruvathukal] not all contributions can be measured by LOC or number of commits

      • implies JOSS guidelines explicitly mentioning "commit history" need updating to include additional sources.
    • Unofficial JOSS comment https://github.com/tqdm/tqdm/issues/887#issuecomment-578571433 and https://github.com/tqdm/tqdm/issues/887#issuecomment-578583505 and contributor comments https://github.com/tqdm/tqdm/issues/887#issuecomment-578558307 and https://github.com/tqdm/tqdm/issues/887#issuecomment-578579211

      • [@gkthiruvathukal] probably deserve more than an acknowledgment for your contributions as significant contributors. I always take an inclusive approach to contributions, no matter how large or small. Hopefully, the lead contributors will chime in with whether we could live with the top contributors as minor co-authors

      • [@gkthiruvathukal] I'd like to see as many authors as possible

      • [@altendky] If I get co-author for my one bug-fix then we should look at everyone else that made non-trivial bug fixes and feature additions and include them as well.

      • [@lrq3000] But we need to define a metric indeed. Personally, I would be very glad to see our fellow co-developers in the author list!

      • I fully agree - if paper authors are defined to be code authors then I insist on including the original Zenodo author list (https://doi.org/10.5281/zenodo.2792998). I do not consider anything non-trivial. Permission need not be sought from all code authors if we go down this route as we would be defining paper authors to be code authors, making the paper an accessory to the library rather than a stand-alone journal publication-with-associated-code. Please note that as-is, simply "liking" or "wanting" this to happen is incompatible with the current JOSS guidelines.
    • Contributor comment https://github.com/tqdm/tqdm/issues/887#issuecomment-578574650

      • [@altendky] Is there something that restricts to co-authors and acknowledgments as opposed to adding a contributors group in between

      • Absolutely agree this is something JOSS should think about.
    • Contributor comment https://github.com/tqdm/tqdm/issues/887#issuecomment-578579211

      • [@lrq3000] contributed the most to this project, maintained it all this time making it a very successful package used in lots of other applications, and made this paper happen. I think we can all agree that he deserves to be the first author [of the paper]

      • I don't think any of these (apart from the "made this paper happen" and "contributed" -- it's irrelevant that it's "the most") are part of the current JOSS requirements.
  3. Address the difference between a) project author(s), b) project maintainer(s), and c) paper author(s):
    • Definitions of all three are required (e.g. "(c) is expected to be the same as (b)," and "(b) is anyone who has ...,").
    • Zenodo author records need not match JOSS author records unless JOSS updates its authorship definition to include all code contributors. It is wrong to expect the other way around (Zenodo to update its authorship definition to include only a subset of contributors).
    • Consider this hypothetical extreme example:
      • A FOSS project contributor responsible for writing a small yet "significant" (assuming a definition is provided and satisfied) amount of all code, documentation, unit tests, etc wishes to write a paper by themselves.
      • Due to the FOSS licence, this person is not required to seek any permission of anyone, nor are they forced to invite co-authorship (otherwise Python maintainers would be invited co-authors of every Python-based library!).
      • By the IEEE definition of authorship, this person will be the sole author of the paper.
      • By the IEEE citation guidelines and by the FOSS licence terms, they will need to cite the project correctly.

2.1) Conclusions

In the absence of any concrete guidance, I can only revert to my own personal code of ethics.

2.1.1) Suggestions

2.1.2) Actions

In the interim period while discussion/decision(s) are ongoing, I will do the following:

  1. Remove https://github.com/tqdm/tqdm/blob/2e58f011c425291d1176ccdf1b2f1ae860ee2dc7/.zenodo.json#L9-L12
  2. Update the table under https://github.com/tqdm/tqdm#contributions.
  3. Continue to maintain tqdm amongst other FOSS efforts.
kmike commented 4 years ago

A good point about authors of paper vs authors of the software. I can see how the whole situation could have arised from different expectations here.

From https://joss.readthedocs.io/en/latest/submitting.html:

Your paper should include: A list of the authors of the software and their affiliations, using the correct format (see the example below).

In the example below a "correct format" is used to define authors with affiliations, which is then shows as authors of the paper. From this I conclude that in JOSS it is expected to have authors of the software as authors of the paper.

https://joss.readthedocs.io/en/latest/review_criteria.html#authorship asks for the list of authors to be reasonably complete. From this it follows that a list of paper authors should be a reasonably complete list of software authors.

Then they ask community to decide what would be a reasonable list of authors. Wording may not be the most precise, but I think then intention is clear. So let's do it :)

How to decide who's an author is subjective, and several approaches are possible. I don't think any legal stuff about authorship matters here. Rules are different in different countries, and we're not in the court. The whole thing is just about fairly acknowledging people for their contributions.

Picking some top contributors, considering their code, discussion and historical contributions looks like a fair way to define an author list. Considering everyone who contributed an author also sounds fine to me. There is also an Acknowledgement section, which can be used to highlight contributions of people who're not in the author list. Alternatively, I think it can be used to highlight top contributors, if "all contributors are authors" approach is taken.

It doesn't look that complicated, after all. Two concrete proposal on how to update the paper, which seems to be in line with the guidelines:

Proposal A.

  1. Put @casperdcl, @lrq3000 and @noamraph as authors.
  2. In the Acknowledgments section put a link to https://github.com/tqdm/tqdm/graphs/contributors and maybe mention that @hadim forked the repo in the dark times.

Proposal B.

  1. Put all contributors to the author list. Getting affiliations can be tricky though.
  2. In the Acknowledgments section say that @noamraph was the original author, @casperdcl has been the main maintainer and contributor in the recent years, @lrq3000 was another prolific contributor, and @hadim forked the repo in the dark times.
lrq3000 commented 4 years ago

@casperdcl , you are shooting yourself in the foot. This situation is your fault, not JOSS's.

This situation arose because you wrote the paper on your own without reaching to any other co-developers, with the exception of me when a JOSS editor requested it. This was evidently an unreasonable procedure.

As you have worked in research, you know that the authors of a paper must include all those who contributed to the experiment. It would be foolish for any of the team member to quickly write a paper secretively, submit it as the sole author, and then claim they are right to do so because noone else contributed to the paper, that's an obvious circular reasoning.

If you had reached to other developers to invite in the writing, and noone would have answered, then it would make for a far better case for you to be the sole paper author. But pre-emptively avoiding contacts barred any other contribution.

So no need for legalese, just common ethical sense. I suggest we stop on this slippery path and come back to fixing the issue at hand. Although @gkthiruvathukal makes suggestions, which I am thankful for, we ultimately are the ones to decide, which is only logical (they only arbitrate).

I agree the Zenodo list of authors can be a good basis, but maybe complemented? Also the order seems subjective, I would like to ensure some degree of objective fairness (if that is at all possible)...

Side-note, about the legal status, if it comes to the worst, the whole project can probably be easily rebased on my minimalist implementation which I made from scratch. I don't know if it still lives here (in the PRs?) but we can certainly find it somewhere.

altendky commented 4 years ago

This implies there are other more minor things you haven't mentioned. Please mention them! Also let me know if I haven't responded to any of your other points.

Even if I did have anything in mind, I'll pass. I stated that you handled my contributions appropriately and what did you do? Rebutted that statement point by point. *boggle* I also commented that while I understand personal preference in workflow that your preference has the presumably unintended side effect of marking PRs as closed rather than merged which can be a bit confusing. Instead of understanding that point you... again, rebutted it.

Please consider not responding to this at all and instead just thinking about the slight feedback on workflow.

[@gkthiruvathukal] This project is lucky to have such a great group of contributors

I find this a highly offensive offhand statement which sounds like a personal attack on all contributors. Luck had nothing to do with it - it took effort.

Clearly that is not an insulting statement. At the same time, there was certainly chance involved in venvs (was it mkenv at the time?) deciding to use tqdm before py2 went eol and capturing the output in the tests thus showing the bug and me being involved enough at that time that I decided to try to fix the issue rather than just dropping use of tqdm, etc.

So, @casperdcl, is the summary of all that you wrote here that you are not clear as to what 'authors' are being discussed and whether they are supposed to be 'the people that wrote the paper' vs. 'the people that wrote the library'? That seems like something that @gkthiruvathukal could probably clear up if it's still in question. Though I haven't been involved in research so I may really be missing something.

lrq3000 commented 4 years ago

[@gkthiruvathukal] This project is lucky to have such a great group of contributors

I am currently reviewing carefully the commits, and I am in awe with what various developers contributed in their free time. They solved lots of small and hard problems I had no idea how to fix at the time. Yes, the success of tqdm is due to a lot of work, particularly by Casper, but it's also lucky that so many found this library useful and interesting enough to dedicate some of their free time to help out. When so many other great projects remain in the dark.

So, @casperdcl, is the summary of all that you wrote here that you are not clear as to what 'authors' are being discussed and whether they are supposed to be 'the people that wrote the paper' vs. 'the people that wrote the library'?

The thing is that the correct process would have been to contact other developers (software authors) beforehand to invite them in authoring/writing the paper (becoming paper authors). Since that was not done and the paper was published, and now on hold, we are in a special case. So now, there is no way to go back and define who should be the paper's author on the criterion on who wrote the paper since this was flawed, and we have to select another criterion. The criterion is ours to choose, JOSS can't choose for us since it's a special case. Or there is always the other possibility of simply retracting the paper if we can't agree, but I think this would be a waste and a loss-loss situation.

We already have several points of agreements. Let's not overcomplicate this, I am convinced we can all reach agreement on a final solution that would content everyone.

gkthiruvathukal commented 4 years ago

Alright, thanks to everyone for your participation. Looking back, I probably shouldn't have chosen the word lucky to describe how your team has such a great group of developers, despite nothing but the best of intentions. The term luck was not by any means intended to slight your efforts but rather the opposite, which many of you seem to have understood. I was doing what many world cultures do by recognizing the importance of being lucky as part of success. We have a phrase in USA, attributable to a famous baseball player (look it up) that says, "I'd rather be lucky than good." (He was onto something.)

Of course, many people mistakenly think that all of the efforts are attributable to their own individual talents, which seems to be at work here. You could also have been unlucky and not had such a great group of people to help evolve the software, but you weren't. It's unlikely that tqdm would be what it is today, if left to the efforts of one person, given that it started with someone who is not the largest committer. Many have contributed to its success, which is important when it comes to the paper/publishing aspect of all this.

For every successful project like yours, there are hundreds that are not. Most projects--in the end--fail for human (not technical) reasons. I hope yours continues to be successful. Resolving the human issues will probably be crucial to ongoing interest in the project.

I'm going to conclude my participation and turn things back over to Arfon. We are sure lucky to have him as our EIC. In the end, I came here to help mediate the situation, and I think I was at least successful in opening the lines of communication and reaching some degree of consensus how to proceed. In psychology, one thing we learn is that getting people to talk is usually the hardest part.

Here are some resources that I hope will be helpful to your project's ongoing success:

On the role of luck, which most underestimate and also fail to acknowledge: https://www.nytimes.com/2017/01/09/your-money/stop-and-acknowledge-how-much-luck-has-to-do-with-your-success.html

A guide for working well with others on open source projects, written by folks who have a lot of experience working on open source projects (developers of Subversion): http://shop.oreilly.com/product/0636920018025.do

Wishing all of you the best!

lrq3000 commented 4 years ago

Thank you again @gkthiruvathukal for your insights and resources, I will surely read them (particularly the book once I buy it!). /EDIT: Oh, I just understood that you were concluding the mediation. Thank you so much for volunteering to help us out, you helped a lot! We will do our best to continue forward and reach an enjoyable agreement for everyone :-)

I have finished my review of the commits. Before writing the results, here is the methodology I have used:

  1. Overview of authors ordered by contribution count: git-fame (great package by @casperdcl) and git shortlog -ns to get a first glance of top authors by loc and by commits count. "Contribution" here is to be understood vaguely, can be loc or commits count.
  2. Overview of commits grouped by author: git shortlog -n to review the description of the commits by each author
  3. Manual deep review: reviewing manually each commits by each author via the github interface. I also identified via github the top authors by number of commits and by number of additions (results slightly differ).
  4. Finding missing authors: in the log of the process of generating the CHANGELOG.MD by github-changelog-generator, there are warnings such as: Warning: PR 671 merge commit was not found in the release branch or tagged git history and no rebased SHA comment was found

I have checked each PR from these warnings and found a few orphaned authors.

  1. Quick checkup of the most commented issues to see if there were "community managers" or another kind of user who significantly contributed apart from code.

Here are the results: apart from the 3 authors mentioned previously (I let you guys decide the order), here are the other software authors that I think should be considered for paper authorship, along with a succinct description of their contribution (feel free to correct me if I missed anything), and in order of significance and effort (so an aggregate subjectively evaluated measure of the commits count without reverts and the content of these commits and how much it made the project evolve) according to me:

* hadim made the initial tqdm logo and initially coordinated the effort towards the new tqdm and merged (casper later refined the logo and animated it) see https://github.com/noamraph/tqdm/issues/18 and https://github.com/tqdm/tqdm/issues/2 and https://github.com/tqdm/tqdm/issues/30 and https://github.com/tqdm/tqdm/issues/33
* Gangshuo Chen major fixes for notebook and pandas + TMonitor minor
* Kyle Altendorf significant changes to output management - note there were lots of reverts, so the commits count does not directly reflect the changes done, but this suggests it was a quite hard feature to implement
* kmike mid, tox setup and various packaging fixes and added the disable keyword, very * important
* obiwanus initial testing and tox setup
* mbargull significantly fixed TMonitor
* mjstevens777 idem
* orivej idem
* martinzugnoni provided an interactive demo ipython notebook
* kcwu significant fix for precision of remaining time calculation (ema)
* charlienewey minor changes to core tqdm, on deduplicating ema calculation
* MPagel significant few changes to tqdm core printing and code organization
* jcea initial arg to restart meter
* yarikoptic lazy loading of modules
* RunOrVeith inf total iterables support
* JackMc support for any file-like output
* tacaswell stack level change of warnings
* CrazyPython community management and copyediting

All the authors above made usually more than 5 commits, some have made only 2 or 3 commits but were significant enough to suggest authorship IMO (example: martinzugnoni, 3 commits 1,656 ++ 476 --, because he added a full ipython notebook demo he made from scratch).

I am not sure whether all these software authors should be in the paper's author list, but I would suggest to include at least down to martinzugnoni (included).

All other software authors that are not listed here, who did minor - but very useful! - changes (which is the primary criterion I have used to differenciate, and who happen to have a number of commits of 3 at most), should I think be thanked in an Acknowledgement sections.

Orphaned authors who should be in the acknowledgements too: cfperez PR#220 jwilk PR#82 enriquefynn PR#76

Also, I think it's important that Noam is credited as the original conceptor of tqdm in the paper.

Finally, here is a Gource visualization of tqdm development history, a nice overview of all the hard work everyone put in:

https://youtu.be/uiOFEbdC1uo

Side-note: please @casperdcl fix this.

lrq3000 commented 4 years ago

Also please let me commend @casperdcl work in improving recognition of contributions, both in the readme, by making the git-fame package, and also by adding badges linking to the contributors.

To further improve recognition of contributions, in line with what gkthiruvathukal proposed and with minimal overhead in development, I would suggest in the future to systematize the use of git-changelog-generator to make and update a CHANGELOG.md file. This will serve both as a change log and an attributon list, since it also includes the authors of each merge (and without relying on github releases nor contributors list).

moshiba commented 4 years ago

Finally, here is a Gource visualization of tqdm development history, a nice overview of all the hard work everyone put in:

https://youtu.be/uiOFEbdC1uo

@lrq3000 I think you accidentally made the video private

lrq3000 commented 4 years ago

@HsuanTingLu Sorry in fact it was still uploading, now it's done, you can watch it :-)

lrq3000 commented 4 years ago

Let's try to wrap this up. As Arfon suggested, and to simplify choice as well as to ensure some form of balance, I suggest me, @casperdcl and @noamraph try to agree on a list of co-authors for the paper.

Here are the 3 options I suggest, note that contributors who are not selected as co-authors will anyway be mentioned in Acknowledgements:

  1. Only @casperdcl , me and @noamraph, this is the minimal common denominator most (all?) participants here agreed on.
  2. In addition to authors in 1, add the following who made significant contributions (significant being defined as > 5 commits or major changes/bugfixes to tqdm, such as adding a full ipython notebook demo or fixing the TMonitor), this would include the following contributors:

Note I have removed orivej, who made minor fixes to TMonitor after I had a 2nd look.

  1. All contributors to the tqdm software will be co-authors.

I have a preference for 2, but I am also ok with 1 and 3 without any preference between the two.

@casperdcl, @noamraph: what do you guys think? :-)

PS: @casperdcl the order and values in this commit look a bit weird to me given what I saw during my code review of contributions, so I'm wondering if maybe LoC in this table is in fact the SLoC (surviving LoC)?

noamraph commented 4 years ago

Hi, I'm fine with all 3 options.

lrq3000 commented 4 years ago

Thank you @noamraph !

@casperdcl what do you think? Do you have a preference?

lrq3000 commented 4 years ago

Hello @casperdcl , did you have some time to take a look at my propositions? Please just let us know your preferences and I'll take care of the practical stuff.

We are about halfway through the deadline Arfon set for us (1 month), I would regret if we couldn't reach an agreement on time.

casperdcl commented 4 years ago

1) tqdm maintenance practices

As I have always stated, the project maintenance is my primary concern, including actively (aggressively? :)) fostering positive feelings by mentioning as many contributors as possible in the docs and websites.

@lrq3000 from https://github.com/tqdm/tqdm/issues/887#issuecomment-579637044:

is great; it would be nice if you can post a script to help with auto-generating this.

from https://github.com/tqdm/tqdm/issues/887#issuecomment-581128956:

the order and values in this commit look a bit weird to me given what I saw during my code review of contributions, so I'm wondering if maybe LoC in this table is in fact the SLoC (surviving LoC)?

The full explanation of how the table is generated is given right above the table (git fame -wMC as of writing).

2) JOSS publication

Hoping to get feedback. Will propose a solution soon.

casperdcl commented 4 years ago

Thanks all for the discussion.

Based on:

I think there is only one solution: think of the "paper authors list" as a "software contributor attributions list."

Proposal

Notes

lrq3000 commented 4 years ago

Thank you @casperdcl for your in-depth look into this issue. I agree with your arguments and your proposition sounds perfect to me.

About whether to use the list at the first submission time, or at revision time (now), since JOSS will publish the new manuscript with an updated time, I think the safest is to use the current revision (and I think this has the advantage to include some late but significant contributors if I remember from the Gource video?).

The only practical issue may be that for some users, we may need to ask them their full name as they did not necessarily set it in their Github profile.

arfon commented 4 years ago

The only practical issue may be that for some users, we may need to ask them their full name as they did not necessarily set it in their Github profile.

We can also publish papers with just GitHub handles (see the authors at the end of this recent paper https://joss.theoj.org/papers/10.21105/joss.01832). It's not ideal, but it's possible.

lrq3000 commented 4 years ago

Dear @mjstevens777, @CrazyPython, @toddrme2178, @aplavin, @Socialery, @RedBug312, @FichteFoll, @zed, @songololo, @littlezz, @immerrr, @igorljubuncic, @elsonidoq, @deeenes, @darindf,

We would like to invite you to be co-authors of a paper about the tqdm library, to which you contributed in the software's code development. In order for us to properly credit you, could you please reply here with your full name?

Thank you very much in advance! Best regards, Stephen Karl Larroque

songololo commented 4 years ago

@lrq3000 my contribution (4 lines) is quite minor so no need to add me, but thank you for checking.

casperdcl commented 4 years ago

Need to clarify here... we're going to add everyone unless they explicitly ask for their name to be removed. "No need to add me" is a bit ambiguous. Please say "Please don't add me." if you want.

Also feel free to post ORCIDs if you have one.

mjstevens777 commented 4 years ago

Please don’t add me. I appreciate the gesture though.

songololo commented 4 years ago

Please don't add me.

zed commented 4 years ago

Please don't add me.

deeenes commented 4 years ago

Thanks for asking, please don't add me, I've done very little contribution

FichteFoll commented 4 years ago

Thanks for asking. If Github handles are fine, then please do add me. Otherwise, please don't.

aplavin commented 4 years ago

Thank you for the kind invitation. My name is Alexander Plavin, ORCID 0000-0003-2914-8554.

immerrr commented 4 years ago

Thank you for asking, please, don't add me, my contribution was quite trivial.

ghost commented 4 years ago

Hello,

Thank you for asking.

Our contribution was small, but if you want, you can credit: Snap Advocacy team.

Regards, Igor

On Mon, Feb 24, 2020 at 11:05 AM immerrr again notifications@github.com wrote:

Thank you for asking, please, don't add me, my contribution was quite trivial.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tqdm/tqdm/issues/887?email_source=notifications&email_token=AKOC5DSHXLHFZOD6AMAKER3REOSWNA5CNFSM4KLUHCT2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMXMUCQ#issuecomment-590268938, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKOC5DWRNSTTHHECLKQW5MLREOSWNANCNFSM4KLUHCTQ .