search

greenelab / meta-review

Manuscript describing open collaborative writing with Manubot
https://greenelab.github.io/meta-review
Other
48 stars 21 forks source link

Evaluate features of Manubot alternatives #44

Closed cgreene closed 6 years ago

cgreene commented 6 years ago

We should check how the features that we discuss relate to the features existing in Overleaf/Authorea.

Authorea + Bibtex (@cgreene will explore) -> Casey Overleaf + Bibtex (@agitter will explore) -> Google Docs + Paperpile (@cgreene) -> Word (Installed) + Ref Management Software (@cgreene)-> Word (Office 365) + Ref Management Software (maybe @agitter) -> Markdown on Github (@cgreene will fill in)

cgreene commented 6 years ago

Proposed features for matrix:

cgreene commented 6 years ago

Something about ease of use might be good to include too. But some of these are questionable ease of use. Like the git-based forms of overleaf may require the knowledge while general use doesn't. But you'd only get character-level attributions if you use the git backend and don't edit concurrently.

agitter commented 6 years ago

I like the features in principle, but when I think about how to fill out the table there is a lot of grey area. For example, Overleaf, Word, etc. allow comments on the text that could support multi-participant conversation, but it would be a very poor way to execute that.

Other feature ideas:

cgreene commented 6 years ago

https://docs.google.com/spreadsheets/d/1AMjzH1a9YB3Y71g0JN-oRQJlGQ-oNjRaHyOJQLEA1aM/edit#gid=0

cgreene commented 6 years ago

☝️ I made a google drive sheet if we want to start filling in there.

agitter commented 6 years ago

We have a complete draft table except for Cite by common identifiers for Google Docs + Paperpile. I assume that is a No, but I haven't used Paperpile.

cgreene commented 6 years ago

I have now filled that in as well. Based both on my experience trying out paperpile + google docs ~6 months ago & perusing the FAQ to see if there were changes.

agitter commented 6 years ago

Great, this looks like it's ready to be added to the manuscript. I suggest that we transpose the table to put the tools in the columns.

agitter commented 6 years ago

We should cross reference our table with Table 1 from http://doi.org/10.22541/au.148769949.92783646, which is similar.

slochower commented 6 years ago

I suggest putting version numbers in the table for the other software. After all, Overleaf or Word Online could add a feature tomorrow that's not there today, and make the table outdate. I'm not sure what the best way to version a Web App like Overleaf or Google Docs, though, so the caption might read "results as accessed by the authors on June 22, 2018" (or something to that effect).

slochower commented 6 years ago
agitter commented 6 years ago

I definitely agree regarding dating the evaluation, especially with Overleaf v2 coming. We can rephrase "Automatic change testing" when we move this into the manuscript.

By manage references, do you mean keep track of relevant references before or during writing? I'm curious how you are using Paperpile alongside Manubot.

slochower commented 6 years ago

By manage references, do you mean keep track of relevant references before or during writing? I'm curious how you are using Paperpile alongside Manubot.

Yes. In my experience trying out a few different reference managers over the past year or so, Paperpile has been the best at extracting and storing accurate metadata. So when I want to keep track of a paper for later, I just use the browser extension to add the paper to my Paperpile database. Then when writing with manubot, I just use Paperpile to search for the paper, select the DOI and cite it with @doi:whatever. AFAIK, when writing with manubot, most people will be using a reference manager alongside (e.g., Zotero, Mendeley, etc.) to keep track of relevant references -- right?

agitter commented 6 years ago

when writing with manubot, most people will be using a reference manager alongside

I'm not sure yet. You are one of the few people using it for real writing, which is why I was curious. For this meta-review paper and deep review, I did not use a reference manager unless I needed to override the CSL JSON. We tracked papers through GitHub issues and could extract the DOI from the issue when it was time to cite them.

However, what you described could end up being the more typical use case. That's likely what I would do if I was writing with Manubot and did not want to open issues to discuss each reference with other authors.

slochower commented 6 years ago

We tracked papers through GitHub issues and could extract the DOI from the issue when it was time to cite them.

Ah right. I knew about that but forgot.

However, what you described could end up being the more typical use case. That's likely what I would do if I was writing with Manubot and did not want to open issues to discuss each reference with other authors.

Also right. But I see reference management and citation management as separate things. Even using Manubot exclusively for authoring and citing, I'd still like to have my references accessible and organized aside from GitHub issues, if only for sharing with colleagues, sending to students, doing full text searches, etc.

cgreene commented 6 years ago

Ahh - this is interesting! When I write with manubot, I don't use a separate reference manager. It had not occurred to me that people might use both!

slochower commented 6 years ago

@cgreene Having a central reference manager makes collaboration easier, in my experience. Sure, I might want to write with manubot and cite articles directly or based on GitHub issues, but if I'm writing related papers with collaborators A using Word and collaborators B using Overleaf / bib files, then having all my references in a central place is the most practical. In fact, I ran into an issue last week where I needed to add references to support a few sentences in a LaTeX file for a collaboration. I considered making a dummy Manubot "manuscript" with just a few citations and no text, running manubot, extracting the references JSON, pulling that into a BibTeX manager like JabRef, and then exporting the new bib file. (I ended up just using PaperPile to export the bib for those entries.)

I apologize that this discussion is pretty far afield from where it started, but I think it was helpful to bring to light a few other use cases for reference management.

slochower commented 6 years ago

We should cross reference our table with Table 1 from http://doi.org/10.22541/au.148769949.92783646, which is similar.

This table has "offline editing," "ease of use," "math support," "output formats," and "interactivity" that are missing in the table @cgreene made. On the other hand, @cgreene's table is more granular. "Editing software" is similar to "offline editing" and some other columns could be considered together to inform "ease of use." We could add a column for math support, but interactivity and output formats might not be useful; once pandoc is taken into consideration, there are way too many to enumerate.

I can convert this table to Markdown and make a PR, but:

agitter commented 6 years ago

I find "ease of use" to be difficult to evaluate, and that may not be an attribute we want to highlight. I agree that math support could be nice to include.

Do we want to loop in any of projects mentioned in #47 with this table?

I'm inclined to leave those out to keep this table focused on the most popular existing platforms that have a reasonably large market share.

Do we want to rename any columns and keep the long entires like "No - conversation occurs on document, not changes."?

Feel free to rename columns in the PR if you think of a better phrasing. For some of the entries a simple "No" didn't seem to justify what we found lacking about the other platform. However, we could also use footnotes in the table caption if we want to keep the text in the table cells shorter.

agitter commented 6 years ago

I'll also evaluate https://doi.org/10.1038/514127a, though these tools have evolved substantially since that was written.

agitter commented 6 years ago

I'm working on a pull request to add this table in Markdown form. Let's stop editing the Google Sheet and make further suggestions in that pull request.