hjhsalo
commented
7 years ago Concept-wise we should consider (at least) following:
-
Should markdown be parsed to html in client-side javascript or at server-side?
-
If parsing happens on the server-side:
- Should markdown be parsed immediately (or multiple records in one go / in batch) after submitting a record and stored to database as html. This html would the be fetched whenever a record is requested. Subsequent parsing of markdown for some record would happen everytime that record is updated.
- Should we store markdown in database and either 1) parse it to html or dictionary before sending to template rendering, 2) or send the markdown to template rendering and let some jinja filter deal with markdown parsing during the actual template rendering.
After we have list of suitable Flask extensions to use or if we decide to implement something simple on our own, we should consider following:
-
Security. We probably don't want to expose html tags other than what is needed for basic-styling of markdown. We should make sure that the selected libraries support stripping of html-tags and possibly in a customizable manner. Or we use a separate library to strip html generated by markdown parser, e.g. Bleach or python implementation of Smartypants.
-
Consistency with popular markdown specifications. We probably want to be as close as possible to some existing, popular extended (opinionated) markdown specification such as Github Flavored Markdown. Also things like support for code-highlighting might be considered important.
-
Performance. Markdown parsers written in Python tend to be a lot slower that something written in C with a pythonic API using CFFI. Markdown parser presented in pythonhosted.org seems to be especially slow compared to alternative such as Mistune. There is also Markdown2 which performance reports compared to Markdown and Mistune seem outdated. Flask-extension for Markdown-parser exists in form of Flask-Markdown and there is one for Mistune as well, Flask-Mistune. As for CFFI based markdown parsers there is Mikasa which has a nice Flask extension as well, Flask-Mikasa. Although some performance reports for python markdown parsers exist ( 1, 2 ), we should take couple of pages / records that we currently have and make our own benchmarks. Note that with CFFI based libraries we probably introduce requirements that are not fully pip-installable.
-
Pythonic API or pythonic way of calling executables through shell. Libraries such as Markdown, Mistune and Mikasa provide a pythonic API to interact with them. On the other hand libraries suck as Flask-FlatPages extension Flask-FlatPages-Pandoc parses markdown (or other markup) by connecting with executables through shell. Flask-FlatPages-Pandoc uses pandoc-executable to parse various markup formats to html. There is also Flask-FlatPages-Knitr which uses Knitr to parse and evaluate R-code. In the same manner we could easily write our own wrapper on some existing tool as well and use that to parse markdown (or other markup) to suitable format for template rendering. I don't know what are the downsides of using shell executable, other than it adds non-pip-installable requirements.
-
Jinja filter support. In order to continue writing with Jinja templates the way they are written right now, the implementation should provide somekind of jinja filter (e.g.
{{ text|markdown }}or{% filter markdown %}). This can be compared to parsing markdown before template rendering, sending the created object (python dictionaries, JSON objects) to template rendering and refering to keys of the submitted object in the template. This might be less important point-of-view in case the planned implementation fulfils other requirements nicely. -
Support for other formats than just Markdown. In the future we might run into feature request that demand support for other markup languages, such as reStructuredText or even mathematic markup presentation such as MathML or Mathjax. We should at least plan how to implement support for a new markup language or new extensions required by content writers? It would be very nice if our final implementation would be extendable to support new languges, but given our short-term goal of COD3-based release, this might not worth it.
-
Support for editing of markdown at opendata-portal. There have been talks about implementing support for in-browser editing and submitting of records written markdown. We should at least plan how this could be implemented. Even a separate client-side javascript library providing a editor could be an option, give that the editor support the same markdown specification that our selected markdown parser takes as input.
tiborsimko
pamfilos
The source code of some static template pages (=guides) or of some dynamic record pages (=news articles) might be written in the Markdown format rather than HTML.
This ticket is about investigating available tools that could help us in displaying Markdown-authored pages in the COD3 ecosystem.
For example, check
Flask-Markdownextensions that would permit to write Flask templates not in Jinja, but in Markdown.For example, check
pandoctools that would read Article record body in Markdown from the DB and render them to users in HTML. (Later, a user could deposit articles via UI, so having a Markdown previewer plugged in the deposit form will also be interesting... but that will come later.)Remember to investigate both scenarios, rendering static guides and rendering dynamic records.