Review of PythonTeX: Fast Access to Python from within LaTeX

[hplgit]

Reviewer: Hans Petter Langtangen

Department/Center/Division: Center for Biomedical Computing

Institution/University/Company: Simula Research Laboratory

Field of interest / expertise: Scientific Computing, Mathematical Modeling

Country: Norway

Article reviewed: See title.

Quality of the approach: meets Quality of the writing: meets Quality of the figures/tables: meets

SPECIFIC EVALUATION

For the following questions, please respond with 'yes' or 'no'. If you answer 'no', please provide a brief, one- to two-sentence explanation.

• Does the article present the problem in an appropriate context? Specifically, does it:
  • explain why the problem is important, YES
  • describe in which situations it arises, YES
  • outline relevant previous work, NO, see comments below
  • provide background information for non-experts YES
• Is the content of the paper accessible to a computational scientist with no specific knowledge in the given field? YES
• Does the paper describe a well-formulated scientific or technical achievement? YES
• Are the technical and scientific decisions well-motivated and clearly explained? YES
• Are the code examples (if any) sound, clear, and well-written? YES
• Is the paper factual correct? seems so
• Is the language and grammar of sufficient quality? YES
• Are the conclusions justified? YES
• Is prior work properly and fully cited? NO, see comments
• Should any part of the article be shortened or expanded? Please explain. NO
• In your view, is the paper fit for publication in the conference proceedings? YES

Please suggest specific improvements and indicate whether you think the article needs a significant rewrite (rather than a minor revision).

Review

PythonTeX will definitely be highly welcome among LaTeX users who uses Python for numerical and symbolic work. The article is well written and easy to follow. As such it can be published as is, but there are two fundamental issues that must be dealt with: the relation to some key references is not discussed, and much of the functionality can be achieved in other systems.

When people think about the desire to have executable code as part of a LaTeX document, they will probably claim that IPython notebooks already offers this functionality. IPython notebooks are not mentioned in the article. Clearly, PythonTeX has considerably more functionality than an IPython notebook, e.g., output and/or code can be suppressed and the values of variables in the code can be accessed directly in the text. However, because of the strong momentum in IPython notebooks today, the author should discuss the pros and cons compared to this tool. Having said this, it is equally natural to mention the many similar the pros and cons compared to Sage notebooks.

It is also natural to discuss the relation to literate programming tools such as

• https://github.com/stdbrouw/python-literate
• http://mpastell.com/pweave/
• http://pywebtool.sourceforge.net/
• http://pythonhosted.org/antiweb/index.html
• http://en.literateprograms.org/Literate_Programming_(Python)
• http://gael-varoquaux.info/computers/pyreport/

Literate programming tools embed the document's text in the program, while PythonTeX and Doconce (below) embed the code in the text.

It appears that much of the functionality in PythonTeX is automatically achieved by using Mako as a preprocessor. This approach is taken in Doconce. For example, the following Doconce code runs the example given in the Step-by-step derivations ... section in the article, where the calculation of a double integral is shown step by step, using SymPy to perform all the mathematics:

# Execute Python code
<%
import sympy as sm
x, y, a = sm.symbols('x y a')
f = a*x + sm.sin(y)
step1 = sm.Integral(f, x, y)
step2 = sm.Integral(sm.Integralf, x).doit(), y)
step3 = step2.doit()
%>

# Make use of results in the above block when writing LaTeX math
!bt
\begin{align*}
${sm.latex(step1)} &= ${sm.latex(step2)}\\
&= ${sm.latex(step3)}
\end{align*}
!et

Blocks between <% and %> are Python code that will be executed, and variables, functions, or modules can be used in the text through syntax like ${module.function(variable)} (which implies a function call, used above to create LaTeX formatting of SymPy expressions).

The result of the LaTeX block above, after Mako is run, becomes

\begin{align*}
\iint a x + y^{2} \sin{\left (y \right )}\, dx\, dy &=
\int \frac{a x^{2}}{2} + x y^{2} \sin{\left (y \right )}\, dy\\
&= \frac{a y}{2} x^{2} + x \left(- y^{2} \cos{\left (y \right )} +
2 y \sin{\left (y \right )} + 2 \cos{\left (y \right )}\right)
\end{align*}

Debugging Python code in Mako is less convenient than debugging Python files directly, so one may prefer to just include the Python code that Mako is supposed to run by

<%
# #include "src/ex1.py"
%>

This is the way this reviewer makes use of SymPy in LaTeX documents to automate the mathematical derivations (i.e., first the SymPy code files are developed and verified, then fragments are included and run in the document such that the text can access the results).

Mako can be used in this way directly with LaTeX; the advantage with Doconce is that LaTeX is only one output format: one can equally well create a range of HTML formats, including Sphinx, or Markdown, IPython notebooks, etc.

Convinced LaTeX users will certainly feel at home with PythonTeX, and it is important for the community to have the key features this tool documented. Therefore, I recommend publication of the article. However, it represents just one technical solution. There are several others, and with Mako one can extend LaTeX as well as any other text format with the capabilities of running embedded Python code and accessing the results directly in the text.

[gpoore]

@hplgit Thanks for the review! I've been traveling for over a week and am just getting back to where I can work on this.

I will be happy to add a short comparison with the IPython notebook. However, at the time this was written 2 years ago (!), the IPython notebook was only about 7 months old, and the nbconvert utility's LaTeX support was in its very earliest stages. If you think more than a short comparison is needed, I could offer a more thorough comparison with the current IPython notebook. But that would require significant revision to reflect the current capabilities of PythonTeX. I am certainly willing to consider doing that, though I have concerns about a 2012 proceedings paper really being a paper from 2014. Perhaps @ahmadia and @stefanv can chime in if they have any particular thoughts about the extent to which papers should be updated to reflect 2014.

I can see how some additional references to previous work, particularly in the literate programming direction, could provide some useful additional context. I will work on adding those. Most of the current references to previous work (Sweave, python.sty, SageTeX, SympyTeX) are more LaTeX-specific and are PythonTeX's more immediate predecessors compared to some of the literate programming tools.

I will also explain a little more explicitly how the PythonTeX approach differs from preprocessors such as Mako.

Thanks again for the comments. I will try to have a pull request with revisions soon.

[ahmadia]

I agree that there is no need for a detailed comparison with the IPython notebook due to the historical context of the paper. We can an editor's note explaining why if you or @hplgit feel that it's warranted.

Also, in your revision, I would appreciate if you could add references to Knuth's approaches to literate programming and Claerbout's early work in reproducible research, which (as far I know) were the two seminal works in their respective disciplines.

[stefanv]

@gpoore Yes, we're doing a bit of strange time traveling here :) Thanks for willing to write a short note on the IPython notebook--that should be sufficient in my opinion.

[hplgit]

All,

I agree that a paper meant to be from 2012 cannot contain technology from 2014, so what is proposed is fine with me.

Hans Petter

On 27 June 2014 02:30, Stefan van der Walt notifications@github.com wrote:

@gpoore https://github.com/gpoore Yes, we're doing a bit of strange time traveling here :) Thanks for willing to write a short note on the IPython notebook--that should be sufficient in my opinion.

— Reply to this email directly or view it on GitHub https://github.com/scipy-conference/scipy_proceedings_2012/issues/13#issuecomment-47296334 .

[gpoore]

I've tried to address everything in #24.

[gpoore]

@ahmadia @stefanv Are there any plans to go forward with the 2012 proceedings, or are they dead? I was just updating a list of publications the other day, and realized that I had completely forgotten about this.

[ahmadia]

@gpoore - They're in a transitional state and waiting on me. Thanks for the reminder.

[ahmadia]

@gpoore: See https://github.com/scipy-conference/scipy_proceedings_2012/issues/31

You can reference: https://github.com/scipy-conference/scipy_proceedings_2012/commit/5f136656bf16b05f0f5be3e022fefd06fa475ba9 or https://github.com/scipy-conference/scipy_proceedings_2012/blob/5f136656bf16b05f0f5be3e022fefd06fa475ba9/papers/geoffrey_poore/geoffrey_poore.rst if you'd like a DOI for the published version of your paper.

A