Peer reviews checklist

[asmeurer]

(feel free to edit this checklist)

editor

• [x] 1. I found the http://live.sympy.org/ site very convenient for trying things out in SymPy, but it is only mentioned in the supplement, which might be overlooked by readers. I suggest moving that section into the main paper, as it provides a great way to play along with the examples while reading the paper.
• [x] 2. The Basic Usage section omits what I think is an important point: how does one distinguish between evaluating exp(1) in Python and exp(1) in SymPy? In other words, how are symbolic constants specified?
• [x] 3. The first thing I tried after seeing Table 2 (simplication functions) did not work:

  trigsimp (exp( Matrix(2, 2, [0, -y, y, 0])) ) fails to recognize cos and sin. Am I expecting too much?

• [x] 4. Section 3.6: are eigenvalues and singular values included? If not, why not?
• [x] 5. Like one of the referees, I expected to see MPFR mentioned in Section 4.1, at least to mention the pros and cons and say why it isn’t used.
• [x] 6. Section 4.1: please state whether the syntax for functions in mpmath is identical, or not, to that in Sympy.
• [x] 7. Unless I have missed something, a weakness of the paper is that speed is mentioned only twice, on lines 496 and 666. My assumption is that SymPy is slow compared with its commercial competitors. Please comment on the speed issue, and not just in the conclusions.
• [x] 8. Editors are needed for [20].

reviewer 1

• [x] 1) A part that confused me is the assertion (footnote 3)...
  • This solved, at least partially (I don't sure if we should explain in details how to define commuatators etc).
• [x] 2) References reasonable, though [8], an excellent reference ...
  • Not sure how to solve this yet. Perhaps, this is a bad reference?

reviewer 2

• [x] 1) >>> circuit_plot(fourier, nqubits=3) ...
  • Notebook reformated (as nbformat 3)
• [x] 2) Line 105...
  • Done.
• [x] 3) Lines 151-153
  • Added a footnote about dotprint
• [x] 4) Section 2.3 - Assumptions...
  • Not sure. Looks like a question about documentation? Or foo.assumptions0? Or foo.is_, i.e. how user can discover assumptions on the object?
• [x] 5) Section 2.4...
  • Suggestion used.
• [x] 6) Line 221:...
  • Done. Footnote moved, perhaps this is better.
• [x] 7) Page 8: Footnote 5
  • Suggestion used.
• [x] 8) Section 4.1... Should mpmath be cited here?
  • Citation added.

reviewer 3

Specific comments:

• [x] 1. Explaining what Python is (lines 70–72) should go before talking about SymPy as a “CAS written in Python”. Further, the paper assumes a moderate familiarity with Python (for example, Python’s console, OOP, and exceptions), and this should be specified. There should be a short note on the used Python console (>>> is the prompt, with the results of computation following immediately in the lines after it). The citation [25] from line 65 should be moved next to “Python” in line 70.
• [x] 2. Line 73 has outdated information. Sage was renamed to SageMath and it no longer aims only at pure mathematics but also at algebra, numerical analysis, etc. The reference [40] should be replaced by a more up-to-date one.
• [x] 3. The plural “CASs” is usually written as “CASes” or “CAS’s”, with the latter being somewhat problematic due to it looking like it implies posses- sion.
• [x] 4. Line 88 mentions “printers”, but it doesn’t state what they are, which is confusing for those readers that are yet to learn the concept in section 3.4.
• [x] 5. Also in line 88, Jupyter’s citation [30] is actually about IPython and should be replaced by a more up-to-date version.
• [x] 6. The word “software” in line 91 is ambiguous; “library” or “package” would make a better choice.
• [x] 7. Lines 91–96: “we discuss/look at/etc” is the preferred form, instead of “section discusses/looks at/etc”.
• [x] 8. The paragraph in lines 103–105 should be moved to the introduction, and the footnote from line 104 should be added to that paragraph as a full-blown sentence, expanded by all the relevant technical information (Python version, OS,...). Given that the end of life for Python 2 is 2020., a comment on whether all the presented examples work in both Python 2 and 3 should be included as well. Further, emphasise that wildcard imports, “import *”, should almost never be used in programs (see PEP 8, the item “wildcard imports”). The same goes for the import mentioned in lines 484–487.
• [x] 9. Line 119 should be removed, as it is basically a copy of the previous line.
• [x] 10. In line 121 the word “stored” should be replaced by “used as keys”.
• [x] 11. Whatdotheauthorsmeanby“therebypermittingfeaturessuchascaching” in line 122? Caching can be done for mutable types as well, just not through hashing. 2
• [x] 12. There is no need to repeat “(CAS)” in line 124, as it was already given in line 64.
• [x] 13. In the same line, the word “represents” should be replaced by “stores”.
• [x] 14. In line 184, “symbols are” should be replaced by “t is” (the general rule is already given in line 179).
• [x] 15. The code in line 210 should be made into its own line (like a displaymath formula), for typesetting reasons and better readability.
• [x] 16. The footnote 4 should be moved from line 221 to line 222, right after “Basic”.
• [x] 17. The part “which defines some basic methods for symbolic expression trees” should be removed from line 222, as it was already given in line 130.
• [x] 18. In line 225, the sentence “Not all SymPy classes are subclasses of Expr.” sounds confusing as a reader new to SymPy wouldn’t expect, for example, symbols to inherit “Expr”. It would be better to expand this, for example “Most of the SymPy classes (including Symbol) are subclasses of Expr, but there are exceptions to this rule”.
• [x] 19. The title “Features” in line 276 is ambiguous, as a “feature” has no precise meaning in Python (or even software libraries in general). It should be replaced by “Packages and modules” or a similar more precise wording. The same goes for ‘feature” in most other places in the paper (for exam- ple, the caption of Table 1. and line 495). It would be very useful to also include actual names of the packages/modules in Table 1., as well as in any section covering those packages/modules.
• [x] 20. The sets support listing is unnaturally split in two by the “This includes...” sentence which would fit better in parentheses.
• [x] 21. Line 355, add a sentence explaining that in SymPy str == repr, because in Python repr is used to get an unambiguous valid Python code repre- sentation, while the return value of str is meant to be human-readable.
• [x] 22. Lines 359 and 379: what is 2D text representation? It seems that “2D” shouldn’t be here.
• [x] 23. Line 427: every dictionary is a “dictionary of keys”. This should be a dictionary with coordinate tuples as keys associated with the appropriate values.
• [x] 24. Section 4 would benefit from an introduction, and lines 440– should be- come a new subsection 4.1. (named “Float” or “Real numbers support” or similar).
• [x] 25. I suggest a better example for lines 459–460: a computation of (e100 +1)− e100 :

   >>> (exp(100)+1).evalf() - exp(100).evalf()
   0
   >>> ((exp(100)+1) - exp(100)).evalf()
   1.00000000000000
   >>> (exp(100)+1) - exp(100)
   1
or two different ways to compute the 100th Fibonacci number:
   >>> phi = (1+sqrt(5))/2
   >>> psi = (1-sqrt(5))/2
   >>> ((phi**100-psi**100)/sqrt(5)).evalf()-fibonacci(100)
   65536.0000000000
   >>> ((phi**100-psi**100)/sqrt(5) - fibonacci(100)).evalf()
   0.e-104

Obviously, like your own example, these are problematic because a part of the computation is relying on Python’s builtin floats. Please include a comment on whether symbolic computation (i.e., applying evalf() on the whole expression) always avoids these errors or not, possibly with an example when it doesn’t resolve this problem.

• [x] 26. The footnote from line 477 should be moved as a sentence in its own right to the introduction, with other technical specifications.
• [x] 27. In line 495, the word “solving” seems more appropriate than “solutions”.
• [x] 28. In line 504 “is” should be used instead of “are” (because “array” is singular).
• [x] 29. In line 518, “produces” should become plural.
• [x] 30. The title of section 5, “Domain specific submodules”, seems inappropriate because the section only covers Physics package (not “submodules”). It should either be expanded with a short introduction listing other domain specific packages, or it should be renamed to “Physics package”.
• [x] 31. The word “symbolics” in line 528 should be removed (as almost everything in the paper deals with symbolic computation).
• [x] 32. In line 530, sympy.physics.vector is a module, not a package.
• [x] 33. It is unclear what “both of these objects” refer to in line 532. My guess is vectors and dyadic objects, but this should be reworded to make it more clear.
• [x] 34. In lines 543 and 545, “rad” should be removed. Radians are assumed when no other measure (like degrees) is given.
• [x] 35. In lines 567–568, “performing symbolic quantum mechanics” makes no sense. This should probably be “computations”, “solving problems related to”, etc.
• [x] 36. The sentence “SymPy expressions are immutable trees of Python objects.” doesn’t belong in the conclusion. This can be moved to the appropriate place when discussing SymPy’s architecture.
• [x] 37. All “submodules” should be replaced by “modules” (examples: lines 657, 662).
• [x] 38. The sentences in lines 657–661 should swap places, because “areas of mathematics” are discrete mathematics, concrete mathematics, etc., while simplifying expressions, performing common calculus operations, pretty print- ing expressions, etc. belongs to common operations (“other areas” is also fine, albeit slightly wrong).
• [x] 39. In line 662 “classical mechanics and quantum mechanics” are listed as the only example of the support for specific domains, as in section 5, which leaves the impression that physics the only one. Either more domains should be listed, or it should be reworded to recognize the fact that there are no others.
• [x] 40. Lines 670–678 contain explanations what some of the authors’ institu- tions are. It is customary to use acknowledgements to thank people and institutions, while institutions’ details should be provided in the authors’ footnotes in the documents’ head.
• [x] 41. The citation [5] in line 691 is missing identification data, probably a URL.
• [x] 42. The citation [21] in line 728 should have “2D” instead of “2d”. More specific comments and correction suggestions for the supplement fol- low.
• [x] 43. The supplement should have an introduction, explaining what is being covered in it in general and in each section.
• [x] 44. Since the Guntz algorithm is covered in depth, it would be good to include how an interested reader can see SymPy’s steps of computation:

     >>> import os
     >>> os.environ[’SYMPY_DEBUG’] = ’True’
     >>> from sympy import *
     >>> x = symbols(’x’)
     >>> limit(sin(x)/x, x, 0)
     DEBUG: parsing of expression [(0, 1, None, None)] with symbol _w
     DEBUG: returned None
     DEBUG: parsing of expression [(_w, 1, None, None)] with symbol _w
     DEBUG: returned ([], [(_w, 1, None, None)], 1, False)
     DEBUG: parsing of expression [(0, 1, None, None)] with symbol _w
     DEBUG: returned None
     DEBUG: parsing of expression [(_w, 1, None, None)] with symbol _w
     DEBUG: returned ([], [(_w, 1, None, None)], 1, False)
     DEBUG: parsing of expression [(_w, 1, None, None)] with symbol _w
     DEBUG: returned ([], [(_w, 1, None, None)], 1, False)
     DEBUG: parsing of expression [(0, 1, None, None)] with symbol _w
     DEBUG: returned None
     DEBUG: parsing of expression [(_w, 1, None, None)] with symbol _w
     DEBUG: returned ([], [(_w, 1, None, None)], 1, False)
     DEBUG: parsing of expression [(0, 1, None, None)] with symbol _w
     DEBUG: returned None
     DEBUG: parsing of expression [(_w, 1, None, None)] with symbol _w
     DEBUG: returned ([], [(_w, 1, None, None)], 1, False)
     DEBUG: parsing of expression [(1, 1, None, None)] with symbol _w
     DEBUG: returned None
     limitinf(x*sin(1/x), x) = 1
     +-mrv_leadterm(_p*sin(1/_p), _p) = (1, 0)
     | +-mrv(_p*sin(1/_p), _p) = ({_p: _Dummy_14}, {}, _Dummy_14*sin(1/_Dummy_14))
     | | +-mrv(_p, _p) = ({_p: _Dummy_14}, {}, _Dummy_14)
     | | +-mrv(sin(1/_p), _p) = ({_p: _Dummy_15}, {}, sin(1/_Dummy_15))
     | |   +-mrv(1/_p, _p) = ({_p: _Dummy_15}, {}, 1/_Dummy_15)
     | |     +-mrv(_p, _p) = ({_p: _Dummy_15}, {}, _Dummy_15)
     | +-rewrite(_Dummy_14*sin(1/_Dummy_14), {exp(_p): _Dummy_14}, {}, _p, _w)
             = (sin(_w)/_w, -_p)
     | | +-sign(_p, _p) = 1
     | | +-limitinf(1, _p) = 1
     | +-calculate_series(sin(_w)/_w, _w) = 1
     |   +-limitinf(_w, _w) = oo
     |   | +-mrv_leadterm(_w, _w) = (1, -1)
     |   | | +-mrv(_w, _w) = ({_w: _Dummy_18}, {}, _Dummy_18)
     |   | | +-rewrite(_Dummy_18, {exp(_w): _Dummy_18}, {}, _w, _w) = (1/_w, -_w)
| |||+-sign(_w,_w)=1
     |   | | | +-limitinf(1, _w) = 1
     |   | | +-calculate_series(1/_w, _w) = 1/_w
     |   | +-sign(-1, _w) = -1
     |   | +-sign(1, _w) = 1
     |   +-limitinf(_w, _w) = oo
     |   +-limitinf(_w, _w) = oo
     |   +-limitinf(_w, _w) = oo
     +-sign(0, _p) = 0
     +-limitinf(1, _p) = 1

Emphasize that the environment variable SYMPY_DEBUG must be set before importing SymPy for the first time.

• [x] 45. In line 183, syntax symbols(a:d) is used without being previously defined or explained.
• [x] 46. Figure 2 should not be a screenshot, but rather proper code with LATEX- rendered results and explanations, as was done in the rest of the paper. A shortened link to SymPy Gamma for the example’s expression can be included for the user to try this for themselves.

[asmeurer]

Regarding reviewer 1, point 2, I agree that [8] is a bad reference, because we are not actually using their definition. The aim was just to have a reference for "simplification is not well-defined", but that paper aims to define it. What is a better reference here? Perhaps Moses [19] from the Carette paper? Perhaps we can remove the reference (the referenced statement is "it must be emphasized that simplification is not a rigorously defined mathematical operation").

[asmeurer]

Regarding reviewer 2, point 4, about assumptions, the best reference for assumptions is http://docs.sympy.org/latest/modules/core.html#sympy.core.assumptions. assumptions0 is internal, and it doesn't list all assumptions anyway. Attributes that start with is_ would be good if we didn't also have all the is_Integer and whatnot. If the best place is the documentation, should we link it in the paper?

[asmeurer]

Some notes on the editor comments (see above):

2: I'm unclear what exactly the editor is asking here. Does anyone have any insights?

3: This indeed doesn't work. We can mention it in the rebuttal. I have opened a SymPy issue here.

(side note, if you want to reference numbers at the beginning of a line in markdown you have to format it in a way that it doesn't think it's a numbering, or else it will renumber them starting from 1)

[asmeurer]

Some notes on reviewer 3 comments (see above):

3: I recall we looked this up and the spelling consensus agreed with the way we have it. We should double check (perhaps we can email the editor to ask if the journal has a convention).

5: We need to check with the Jupyter folks on what their preferred citation is.

7: We had decided not to use first person. We should perhaps email the editor to ask about this.

9: Line 119 is the output of the input from line 118 (for reference, there are the lines)

118 >>> (x**2 - 2*x + 3)/y
119 (x**2 - 2*x + 3)/y

so not showing it would be incorrect. The aim here was simply to show that the input expression remains unevaluated. If anyone has any suggestions on how to improve this presentation, let me know. Perhaps we can mention that the output remains symbolic. At any rate, this also seems worth a mention in the rebuttal letter.

13: Not sure I agree with this, but it's a minor point.

16: This was already fixed in https://github.com/sympy/sympy-paper/pull/203 via reviewer 1 point 6 (not that exact place, but I think it is equivalent)

22: Let's replace "2D" with "multiline" or something similar (or explain what is meant by 2D).

25: I'm unclear why this example might be better. I suppose it is simpler. Perhaps @fredrik-johansson can comment if he approves of this change.

37: I disagree with this. "submodules" specifically indicates that they are subpackages (in the Python sense) of the sympy top-level package. The Python documentation seems to agree with this convention (see https://docs.python.org/3/tutorial/modules.html#packages and https://docs.python.org/3/glossary.html).

41: There is a url in the bib entry. How do we make it appear in the citation?

[asmeurer]

Regarding editor comment 8, the editor for the article is in the bibtex file. The bibliographystyle used by the template is apalike.

Regarding the citation for Jupyter, I have asked them here.

[asmeurer]

Regarding the list of assumptions, the way to do it in code is sys.modules['sympy.core.assumptions']._assume_defined (you can't even use sympy.core.assumptions._assume_defined because of sympy.core.core). Clearly this isn't something we want to recommend in any way, especially when there is a page in the documentation.

[ashutoshsaboo]

Hi @asmeurer . I guess we must also move the SymPy Live section to the main paper. It'll provide a very good base for testing all the examples that we give ahead (in the paper) further.

[asmeurer]

I would just mention SymPy Live (a single sentence), perhaps in the intro or at the top of the architecture section near the first example.

[asmeurer]

Regarding SageMath, http://www.sagemath.org/library-publications.html#CiteSage and https://wiki.sagemath.org/Publications_using_SageMath?action=show&redirect=Publications_using_SAGE give a bibtex entry. It is a software citation, which would mean replacing the existing paper citation with a software citation, which is somewhat unfortunate, but I think it's fine. It also says we should contact them (I guess so they can put it on that page), so I will do that as well.

[asmeurer]

I guess we just need to make a PR against https://github.com/sagemath/publications/

[asmeurer]

Regarding reviewer 3, point 46 (about the SymPy Gamma screenshot), what do people think? Sergey has added a note that content is rendered as html, which I agree with (we should not re-render it as LaTeX). But we also might want to just remove it altogether. It does make the technical aspects of submitting the paper easier if there are no figures 😉