TEST: Setup Testing Infrastructure for RST to IPYNB

[mmcky]

This PR sets up the infrastructure for testing rst to ipynb conversions. I have put this test infrastructure at the root level of the repository in tests/ under the idea that they live outside of the standard python testing framework such as pytest which should live sphinxcontrib/jupyter/tests/.

Next Steps:

• [x] add nbdime test command to Makefile
• [x] see if can integrate into a continuous integration framework which would install sphinx, run make clean, then make jupyter to generate the notebooks and make test to compare _build/jupyter/*.ipynb with ipynb/*.ipynb

[mmcky]

@vidartf you might have a quick answer for this question about nbdime. From my testing today the exit code for nbdiff is 0 regardless of whether there are (or are not) differences in the notebooks. I am just wondering what the best way would be to test for the condition that there are no differences between two notebooks.

[mmcky]

• [x] specify a smaller set of dependencies and remove the full anaconda install.

[mmcky]

I have used nbformat and nbdime.diffing.notebooks to check for notebook differences in check_diffs.py. The Makefile can run this script to check newly compiled ipynb output against a collection of REFERENCE notebooks located in tests/ipynb

[mmcky]

Migrating test case files from https://github.com/nthiery/rst-to-ipynb/tree/master/tests. (Thanks @nthiery)

• [x] code_blocks.rst
• [x] images.rst (moved hood.jpg to _static/)
• [ ] linalg.rst (not migrated as causes error as reported in #29 and doesn't compile)
• [x] links.rst (currently issues sphinx warnings, update test as link support is provided through the extension)
• [x] lists.rst
• [x] math.rst
• [x] quote.rst

Not Migrating

• topic.rst - perhaps this is a custom directive from sage or rst2ipynb

TODO

• [x] need to add reference ipynb notebooks of what these should be compared against as these test cases will currently fail.

[vidartf]

From my testing today the exit code for nbdiff is 0 regardless of whether there are (or are not) differences in the notebooks.

The simplest way is to check for output in stdout. If it is empty, the requested diff is empty.

[mmcky]

Great - thanks @vidartf.

[vidartf]

That being said, I think your decision to use nbdime as a library is better. You get more control about the nuances. See e.g. nbdime.diffing.notebooks.set_notebook_diff_targets(), either for use or as inspiration.

[mmcky]

Ah - I see - Thanks for highlighting this. It might be good to be able to switch off checks on metadata etc. https://github.com/jupyter/nbdime/blob/e9ba0e490e40892b207de75fa0726dfc4848b52e/nbdime/diffing/notebooks.py#L530

[mmcky]

Moved rst2ipynb test cases to its own folder and will work on migrating them in one by one and reviewing each for differences.