search

wshanks / Zutilo

Zotero plugin providing some additional editing features
Other
1.53k stars 72 forks source link

Switch the documentation source from HTML to reStructuredText or Markdown #20

Closed acsr closed 10 years ago

acsr commented 10 years ago

Working in HTML directly for writing docs is quite hard. I am using Sphinx http://sphinx-doc.org/ and reStructuredText to create my docs and it is much more efficient and readable.

We can transfer the existing HTML in one step _[1] to reStructuredText or Markdown using pandoc. Both markups are supported by github to render the pages online. I can do this experimentally in the german translation.

.. _[1] Important for all languages after they are in one state to keep diffs working from that point on.

Personally I would prefer reStructuredText because you can use then use Sphinx to generate the HTML and a documentation website automatically and independent from githubs rendering. But that may interfere with other workflows so I am comfortable to earn as well.

acsr commented 10 years ago

Experimental conversion to rst done with pandoc 1.12.2.1. you can find it at https://github.com/acsr/Zutilo/blob/locale-de/chrome/locale/de/zutilo/README.rst . The rendered html may be sepereated from the rst source depending on the requirements for the Zutilo xpi packages. I am not familiar with this.

acsr commented 10 years ago

Since the main README seems to be already Markdown, the HTML in the translation may come from the babelzilla history. So I updated my fork also with a Markdown version.

Please let me know which version you prefer to continue!

acsr commented 10 years ago

here the pandoc commands for reference:

cd ./Zutilo pandoc -o chrome/locale/de/zutilo/README.rst chrome/locale/de/zutilo/README.html pandoc -o chrome/locale/de/zutilo/README.md chrome/locale/de/zutilo/README.html

acsr commented 10 years ago

Update: One thing I now recognized, is that the pandoc conversion does word wrap on lines.

This is suggested by rules like PEP8 in Python but it is not a good idea for version control. Every change in sentences that results in changing the line wrap messes up any further diff.

For translations and manuscripts the following rule is better: Start every sentence in a new line. One more benefit is that tools like transifex can identify indetical sentences and translate them in .po file automatically and offer autocomplete.

For further arguments read: http://rhodesmill.org/brandon/2012/one-sentence-per-line/

So I need to check the pandoc options to switch off word wrap or fix them initially by hand before working on.

wshanks commented 10 years ago

Thanks for the suggestions here and in issue #19.

I had been using markdown for the README because that was the default format that GitHub suggested. We can go with whatever will make the translation workflow easiest. I like the idea of breaking the sentences down out into their own lines. Let me know if you find something in the pandoc documentation that will make this easy, or if using Sphinx would make this easy.

I have another folder with peripheral files to addon. It has the screenshots for the addons.mozilla.org listing, the script I use to convert readme.md into readme.html, the script I use to convert the readme.html files into the format that is allowed on addons.mozilla.org, and the script I use to create the zutilo.xpi file. I will organize that folder and add it to the repository so that whatever script(s) we need for creating the documentation from the itemized translations can be saved there.

acsr commented 10 years ago

today I figured out the options to preserve paragraphs and disable word-wrap during pandoc conversion. "--no-wrap" disables text wrapping in output.

Additional I will check if the option --normalize will increase the quality.

--normalize Normalize the document after reading: merge adjacent Str or Emph elements, for example, and remove repeated Spaces.

I will first update the README.rst and README.md documents in the repo for review of the quality after doing it myself. Then let you @willsALMANJ decide how to replace the source file entering your release workflow.

acsr commented 10 years ago

commands::

cd ../myfork/Zutilo/chrome/locale/de/zutilo
pandoc -f html -t markdown --no-wrap --normalize --tab-stop=4 README.html -o README.md

or::

pandoc -f html -t rst --no-wrap --normalize --tab-stop=4 README.html -o README.rst

Splitting into lines after fullstops was done by hand.

For this I need:

It seems there was some rework of the DE docs inbetween, but I am not sure because of the switch from babelzilla.

Cheers Armin

acsr commented 10 years ago

to check latest changes see latest commits in my fork at https://github.com/acsr/Zutilo/commits/locale-de/chrome/locale/de/zutilo/README.md ::

git diff 5a46823e2a d1903ef22f

I think it needs some futher review before issuing a pull request

wshanks commented 10 years ago

Thanks, Armin. That looks great. If you look at the diff in GitHub with the "rendered" view, it highlights only the modified words (rather than the whole lines) which is really convenient.

I think it is probably fine for me to pull in your md file at this point. I need to decide on how to organize these external files (since only the rendered html gets bundled into the extension), but that shouldn't be hard. It looks like some of the formatting was slightly changed in going from english md -> english html -> german html -> german md, but I can change that back by hand.

I am not sure about the things you say you need. You mean that you want to generate updated docs for versions 1.2.3 through 1.2.9? I wouldn't bother with that. As far as I know, there were backwards incompatible changed within those versions, so there is no reason for anyone to use them other than that they don't feel like updating and in that case they won't see the updated docs any way.

acsr commented 10 years ago

OK, so my questions are:

A. go on in my fork to update the docs too the latest EN version.

or

B. do I wait until you fixed the german md back to intended formatting. Then update my fork and rework from this.

or

C. You take over my current md, merge it as is !!important!! and then fix the formatting yourself. Then update to the current version (and I do a review again in the diff).

I would prefer C. since your translation is not completely unreadable but only missing some "final style optimization." If you prefer A or B, please send me the prepared link for the diff, because I cannot understand exactly what you did.

wshanks commented 10 years ago

I will do C. I will use a separate translation branch and merge back into master once we are both happy with the docs.

Sorry, my free time has been pretty light recently, so I haven't had much time to work on Zutilo recently. I will keep looking for spots to work on it.

acsr commented 10 years ago

Ok, I wait until pinged by you

wshanks commented 10 years ago

Okay, take a look at the new translation branch. I reformatted your README.md and moved it to a new location under the Extra folder. I put in some temporary blank lines to try to make the line numbers match the English README.md better (though still not perfectly) as well. There are several other new files as well. Hopefully, the structure is clear. Check out the Extra/docs folder (very rough) for a few notes on how things are structured as well.

wshanks commented 10 years ago

Just to let you know -- I uploaded Zutilo's locale files to Transifex, so I could test it out and see if it's easier to use than BabelZilla. I had to insert non-empty lines for each line of text in README.md for it to accept the translations as matching the source file, so I put lines with a single "." as placeholders where the English version had a string that the translations did not. If you want to look at it, it's at https://www.transifex.com/projects/p/zutilo/ It may or may not be easier to use than whatever workflow you have for editing the text files locally.

acsr commented 10 years ago

@willsALMANJ due to a serious accident of my wife my sparetime for such tasks is currently almost zero. I requested joining the transifex group ... Let's see

wshanks commented 10 years ago

No problem, Armin. No pressure from me for you to do anything, though it is good to hear an update from you. I am sorry to hear about the reason you haven't had time though. I hope your wife is recovering. You will notice there hasn't been much activity on Zutilo recently. Every weekend it (or one of my other Zotero projects) is what I plan to work after completing my errands but I end up not getting much done on it.

acsr commented 10 years ago

@willsALMANJ I finished review of the german translation in Transifex (except the main readme), and fixed ever issue I found. The workflow is easy for me. The only issue I see is that notification/commenting is only possible "per sentence". Due to that I just give you notification here.

I think you can close this ticket now.

Please reopen a new one related to changes in the main docs (Readme) when an update is necessary.

Currently a complete review of the updated Readme is necessary. Due to the connection sentence by sentence in Transifex it should be much easier than in a text editor.

wshanks commented 10 years ago

Thanks, Armin. I'm closing this one now since I think we are agreed on the formatting of the documentation source.