Reviewer 2 comment 18

[agitter]

Table 1. → You should add a WYSIWIG feature that is present in a number of platforms and important for a lot of non technical people. Same for inline comments and diff colorization.

[vincerubinetti]

I'd agree with this. Perhaps it could go in "Limitations" as well.

Would a mention of the Hypothesis annotations plugin somewhere make sense? Would that address "inline comments"? Also could you elaborate on "diff colorization"?

[agitter]

What are our thoughts about whether Manubot has WYSIWIG support? It is possible to preview the edited markdown and render elements like lists and formatting. However, the figure references, citations, and other aspects are not WYSIWIG.

I'm not sure about inline comments. It could refer to the other platforms and which of them support commenting on specific lines. Most of them do. In general, this entire table may need a refresh. We should switch to Overleaf v2 now that v1 is no longer available (#27).

We should review our text to make sure the inline commenting on Manubot manuscripts through Hypothesis is clearly stated.

For diff colorization, my interpretation is that this would annotate whether the writing platform can diff text updates and color the changes, similar to the GitHub pull request web interface.

[slochower]

What are our thoughts about whether Manubot has WYSIWIG support? It is possible to preview the edited markdown and render elements like lists and formatting.

Is it worth stating that many editors like Atom, VS Code, Sublime... can do interactive preview of Markdown documents? On the one hand, that is not really Manubot-specific in any way, but on the other hand, if not everyone knows this and they are considering using Manubot, it could be helpful.

We should review our text to make sure the inline commenting on Manubot manuscripts through Hypothesis is clearly stated.

Does anyone use Hypothesis to comment during the drafting process? Do comments in Hypothesis get preserved across pushes to gh-pages?

[dhimmel]

What are our thoughts about whether Manubot has WYSIWIG support?

There is no WYSIWIG editor and it is not on the roadmap. Of course, you can use a partial WYSIWIG solution or a preview solution, but it won't understand the full array of formatting options. If you need WYSIWIG, something like Manuscripts App 2.0 (not released yet) or another browser based editor probably makes the most sense. I think this is a limitation of Manubot for the time being, but one of the reasons we have a specific target audience in mind that is familiar with writing source text files.

Same for inline comments and diff colorization

For inline comments, I think Hypothesis and PR line comments are sufficient. Regarding diffs, you can compare markdown or HTML documents using git, you can compare versions of DOCX, and we're working on better rendered diffs in https://github.com/greenelab/manubot-rootstock/issues/54 (although the timeframe on this is unknown).

Does anyone use Hypothesis to comment during the drafting process?

Yes check out https://trang1618.github.io/tpot-ds-ms/ where coauthors used Hypothesis to provide feedback and review.

Do comments in Hypothesis get preserved across pushes to gh-pages?

AFAIK, they will stick to whatever URL they were created on until the text they annotate no longer exists. Therefore, it maybe makes sense to annotate a versioned URL, such that the annotations will never break. Especially if they are annotations that are for a specific version and shouldn't necessarily last forever.

[slochower]

There is no WYSIWIG editor and it is not on the roadmap.

We could mention autobuild.sh. That's about as close as we can come. We could make an animated gif showing a user writing in vim and autobuild displaying the output. Although that really only demonstrates half the pipeline (i.e., not the GitHub and CI portion), it is close to WYSIWIG writing.

[agitter]

On a related note, we may want to ask Overleaf and Authorea representatives to review Table 1 after we update it to make sure we're being accurate and fair.

[agitter]

If we could push artifacts as in #121 that would also help. Still not WYSIWIG, but it would be easier to catch errors.

I could work on this if I have time, but I'm going to tackle all the other issues I self-assigned first. Can anyone else work on this table and review the features of competing systems again? I can comment on those I use regularly, like Overleaf v2.

[agitter]

This is one of the only reviewer comments that we don't have a plan to resolve. Can we divide and conquer updates to the table, with each of us taking a competing system or two?

[slochower]

Sure, you can assign one to me.

[agitter]

I recommend that our revisions here update Overleaf v1 + BibTeX to Overleaf v2 + BibTeX because there were substantial changes. Then we can add the following rows and update all of the columns:

• WYSIWYG: is the platform a WYSIWYG editor or does it have a native WYSIWYG mode? We'll need to introduce this acronym in the paragraph before the table.
• Inline comments: does the platform support inline comment bubbles that appear on the manuscript or rendered manuscript? That is, I don't think LaTeX comments should count.
• Diff colorization: We need to scope this more specifically or use multiple responses. One way to pose the question is can users view a diff of the source or the rendered manuscript? We could use Source, Manuscript, and Source and Manuscript as responses here. Or Source diff, Track manuscript changes, etc. Tracking changes in Word is not exactly a diff, but we should acknowledge it.

@slochower could you please take Authorea + BibTeX?

~@dhimmel or @vincerubinetti could you please take Manubot and Markdown on GitHub?~

~@vsmalladi would you be able to work on Google Docs + Paperpile and Word Online?~

I'll update all features for Overleaf v2 + BibTeX (#27).

Rather than orchestrate 3-5 separate pull requests, we can organize the updates here and make one pull request. We can introduce more footnotes if some of these answers are not straightforward for some platforms.

We can update that date of evaluation to say something like

We initially assessed features on June 15, 2018 using the free version of each platform and updated our assessment on April X, 2019 to add Overleaf v2 and columns A, B, and C

Finally, after we make these updates I'd like to ask Overleaf, Authorea, etc. to make sure we are being accurate. Maybe via Twitter?

[slochower]

~@agitter How do you want to handle the PR with regards to updating the Table. Should I create my own PR to update the Authorea + BibTeX row?~

Sorry, totally missed your comment:

Rather than orchestrate 3-5 separate pull requests, we can organize the updates here and make one pull request. We can introduce more footnotes if some of these answers are not straightforward for some platforms.

I'll update this comment with results for Authorea.

---

• WYSIWYG: Yes (Reasoning: Authorea renders LaTeX quickly & inline, and also supports a lot of text formatting options, e.g., bold shows up bold.)
• Inline comments: Mostly, but might depend on template (Reasoning: In a "Scholarly Article Template" you can attach comments to a mouse highlight and the viewer can choose to enable or disable the comments pane. Even though the selected text is quoted in the comment, visually the comment looks like it applies to the whole paragraph. In the "Scientific Paper" template, which uses LaTeX, selecting text with the mouse puts me in edit mode, so comments are limited to the whole paragraph level based on my limited tested. Also, there is a git bridge, but and then you can comment on, e.g. GitHub, but since this is outside the Authorea interface, I'd say it doesn't count.)
• Colorization of changes: Yes (Reasoning: In the built-in "History View," changes by authors are highlighted.)

[agitter]

Thanks @slochower. I just created WIP #188 to start collecting these feature summaries.

Regarding Authorea:

• WYSIWYG: Are those text formatting options (e.g. bold text) still controlled with LaTeX commands? I'm interested in contrasting it with Overleaf v2, which has a Rich Text mode that adds formatting without requiring the user to know the LaTex commands (screenshot below). We could also say No, but rapid preview or something short that fits in a cell.
• Inline comments: Shall we summarize this as Yes or would you like to describe the limitations as well?

I'm not sure what to call the Colorization of changes feature. We can decide that in #188 once we see what different platforms offer.

Overleaf Rich Text:

[slochower]

Are those text formatting options (e.g. bold text) still controlled with LaTeX commands?

In "Scholarly Article Template", no I don't think so (it behaves just like a rich text editor where you can't see what happens under the hood, but the words just become bold). In "Scientific Paper", yes, you can see that the bolding action inserts \textbf{} surrounding the highlighted text. Scholarly Article Template behaves like Overleaf's Rich Text mode but, as far as I know, it's not based on LaTeX and therefore you can't see what's going on. Does that make sense?

Inline comments: Shall we summarize this as Yes or would you like to describe the limitations as well?

I think Yes is fine. We don't need to split hairs unless there is a reason.

[agitter]

We don't need to split hairs unless there is a reason.

I agree with that policy. Many of these are in a gray area, and we can err on the side of being generous.

I'll use Rich text available as the initial entry for Authorea and Overleaf in the WYSIWYG row.

[agitter]

I'll finish the remaining platforms. These new features are not that hard to update. #188 will need a careful review when it's ready.