Integrate aumayr's beancount-docs into beancount/doc

[tbm]

Original report by Martin Blais (Bitbucket: blais, GitHub: blais).

---

https://github.com/aumayr/beancount-docs

[tbm]

Original comment by Dominik Aumayr (GitHub: aumayr).

---

@blais Now that beancount is mirrored at github.com/beancount/beancount, it would be possible to automatically publish the HTML-version at beancount.github.io/beancount

[tbm]

Original comment by Martin Blais (Bitbucket: blais, GitHub: blais).

---

Yes, I'd love to do that. While I still think the GDocs docs are a great authoring and collaboration system, I think some people are put off by it, and I'd like to integrate what you did as reference docs with my download scripts, converting all the docs to Markdown and then to that HTML format as a one-way conversion that I can run either manually regularly, or in an automated fashion.

[tbm]

Original comment by Dominik Aumayr (GitHub: aumayr).

---

Perfect! I also think that Google Docs is a valid solution [1].

Please let me know if I can help!

[1]: I have been thinking on multiple occasions about implementing a documentation system as a hybrid of Google Docs (WYSIWYG, Commenting) with "traditional" systems (Markdown, Versioning).

[tbm]

Original comment by Martin Blais (Bitbucket: blais, GitHub: blais).

---

I had another idea previously, which would be to wrap all the Google Docs document in an iframe so that they would render in "view-only" mode by default, and not be editable; one could then render an Edit button that would have taken the user to the real Google Docs page for editing. This would alleviate the "finger fart" I'm always having to deal with -- people doing accidental edits and then I'm getting email about it. I think I'll abandon that idea for now, it's a lot of work, and could be an entire other project IAOI.

I think what we ought to do now, is something closer to what you started: that is, to use this code I wrote to download PDF files: https://bitbucket.org/blais/beancount/src/tip/src/python/beancount/docs/download_docs.py but instead of fetching the PDF version, I'd like to test out fetching the Markdown version (or whatever closest is available, I think there are text and rtf versions that we could convert to Markdown or rst or whatever Sphinx accepts), and trying to run Sphinx on that to see how close to the original documentation it comes out. If we could render those Google Docs into the Sphinx placeholder pages you created, that would be the optimal outcome.

If we could do that and integrate your docs generation into a single Sphinx output, that would be the perfect conversion. We could move that into the beancount.github.io page and call it a day. If you'd like to help with this, feel free to modify the download_docs scripts and experiment.

[tbm]

Original comment by Dominik Aumayr (GitHub: aumayr).

---

I recently came across https://docs.landscape.io/, which is a beautiful layout (colors and fonts could be changed) for a documentation-frontpage IMHO:

The code/configuration for Sphinx is here: https://github.com/landscapeio/landscape-docs

[tbm]

Original comment by Kirill Goncharov (GitHub: xuhcc).

---

Hi!

What's the status of this issue?

I wrote a simple script for conversion from Google Doc to markdown (using pandoc), and here is the result for "Beancount Language Syntax" document: https://pastebin.com/raw/n4ginCs8

Indented code blocks are not converted properly but I think it could be improved.

[tbm]

Original comment by Martin Blais (Bitbucket: blais, GitHub: blais).

---

There’s ongoing code here: https://bitbucket.org/blais/beancount/src/tip/experiments/docs_rst/

An an associated repo to work from:
https://bitbucket.org/blais/beancount-jsondocs/src

But really, what ought to be done is to use the new Google Docs API to process the document structure properly:
https://groups.google.com/d/msg/beancount/LoUrIe3mqHs/DSP7D1l9BQAJ

There’s a bunch of other threads on the list.

[tbm]

Original comment by Kirill Goncharov (GitHub: xuhcc).

---

Oh, I didn't know about these scripts.

I created my own and achieved satisfactory results with the following steps:

1. Export document as *.docx (GDocs API is not needed for this: https://docs.google.com/document/export?format=docx&id=1wAMVrKIA2qtRGmoVDSUBJGmYZSygUaR0uOMW1GV3YE0)
2. Find all paragraphs in Consolas font and apply SourceCode style to them with python-docx (pandoc treats them as code blocks and preserves indentation).
3. Run pandoc to convert from docx to markdown, do additional processing with pandoc filter.

Example: https://pastebin.com/raw/rd6btjGa

The only remaining issue is indents in TOC.

Does it make sense to continue work on this approach?

[tbm]

Original comment by Kirill Goncharov (GitHub: xuhcc).

---

I published my code here https://github.com/xuhcc/beancount-docs

[tbm]

Original comment by bradyt (Bitbucket: bradyt, GitHub: bradyt).

---

We might be able to get Kirill’s output to work with MkDocs, a static wiki generator. A prototype is at https://bradyt.github.io/beancount-docs/. With a few small commits on top of Kirill’s work, we have a prototype with toc working on first two pages, an image working on the second page, footnotes are working, “edit page” is working, search is working.

I think Kirill or I will see if we can fix the remaining issues with toc and images, with some minimal steps.

Aumayr, does this give you any ideas? I think you’re currently working on Sphinx wiki generator, and experimenting with themes beyond ReadTheDocs. My understanding is, MkDocs is easier up front, and maybe not as flexible as Sphinx, but on the other hand, as Kirill points out, it’s Python.

Please ignore my personal preference of reenabling wrapping.

I think I’ve read somewhere an idea that Beancount Google Docs might be modified to improve how it is parsed by downloaders. Maybe that would improve things here.

[tbm]

Original comment by Kirill Goncharov (GitHub: xuhcc).

---

Thanks @{557058:348f01fd-fedc-4974-af67-201c1a83a976}!

Documentation website is now generated using mkdocs (I changed theme to readthedocs): https://xuhcc.github.io/beancount-docs/

[tbm]

Original comment by Martin Blais (Bitbucket: blais, GitHub: blais).

---

You guys are amazing.