search

computationalmodelling / nbval

A py.test plugin to validate Jupyter notebooks
Other
443 stars 51 forks source link

Test failure with matplotlib 3.4: to_png deprecated #167

Open bnavigator opened 3 years ago

bnavigator commented 3 years ago

Similar to #155, a new warning message of matplotlib results in a test failure:

[   43s] =================================== FAILURES ===================================
[   43s] _ test_print[/home/abuild/rpmbuild/BUILD/nbval-0.9.6/tests/ipynb-test-samples/test-latex-pass-correctouput.ipynb] _
[   43s] 
[   43s] filename = '/home/abuild/rpmbuild/BUILD/nbval-0.9.6/tests/ipynb-test-samples/test-latex-pass-correctouput.ipynb'
[   43s] correctoutcome = 'pass'
[   43s] 
[   43s]     @pytest.mark.parametrize("filename, correctoutcome", testdata, ids=testnames)
[   43s]     def test_print(filename, correctoutcome):
[   43s]     
[   43s]         command = ["py.test", "--nbval", "-v", "--current-env", filename]
[   43s]         print("Starting parametrized test with filename={}, correctoutcome={}"
[   43s]               .format(filename, correctoutcome))
[   43s]         print("Command about to execute is '{}'".format(command))
[   43s]     
[   43s]         if os.name == 'nt':
[   43s]             exitcode = subprocess.call(command, shell=True)
[   43s]         else:
[   43s]             exitcode = subprocess.call(command)
[   43s]     
[   43s]         if correctoutcome == 'pass':
[   43s]             if exitcode != 0:
[   43s] >               raise AssertionError("Tests failed on ipynb (expected pass)")
[   43s] E               AssertionError: Tests failed on ipynb (expected pass)
[   43s] 
[   43s] tests/test_unit_tests_in_notebooks.py:68: AssertionError
[   43s] ----------------------------- Captured stdout call -----------------------------
[   43s] Starting parametrized test with filename=/home/abuild/rpmbuild/BUILD/nbval-0.9.6/tests/ipynb-test-samples/test-latex-pass-correctouput.ipynb, correctoutcome=pass
[   43s] Command about to execute is '['py.test', '--nbval', '-v', '--current-env', '/home/abuild/rpmbuild/BUILD/nbval-0.9.6/tests/ipynb-test-samples/test-latex-pass-correctouput.ipynb']'
[   43s] ============================= test session starts ==============================
[   43s] platform linux -- Python 3.8.8, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3.8
[   43s] cachedir: .pytest_cache
[   43s] rootdir: /home/abuild/rpmbuild/BUILD/nbval-0.9.6
[   43s] plugins: nbval-0.9.6, cov-2.11.1, timeout-1.4.2
[   43s] collecting ... collected 2 items
[   43s] 
[   43s] tests/ipynb-test-samples/test-latex-pass-correctouput::ipynb::Cell 0 PASSED [ 50%]
[   43s] tests/ipynb-test-samples/test-latex-pass-correctouput::ipynb::Cell 1 FAILED [100%]
[   43s] 
[   43s] =================================== FAILURES ===================================
[   43s] _____ tests/ipynb-test-samples/test-latex-pass-correctouput.ipynb::Cell 1 ______
[   43s] Notebook cell execution failed
[   43s] Cell 1: Cell outputs differ
[   43s]                                                                                                                                                
[   43s] Input:                                                                                                                                         
[   43s] x**2 + y**3
[   43s] 
[   43s] Traceback:
[   43s] Unexpected output fields from running code: {'stderr'}
[   43s] 
[   43s] =============================== warnings summary ===============================
[   43s] tests/ipynb-test-samples/test-latex-pass-correctouput.ipynb::Cell 0
[   43s]   /usr/lib/python3.8/site-packages/jupyter_client/manager.py:66: DeprecationWarning: KernelManager._kernel_spec_manager_changed is deprecated in traitlets 4.1: use @observe and @unobserve instead.
[   43s]     def _kernel_spec_manager_changed(self):
[   43s] 
[   43s] -- Docs: https://docs.pytest.org/en/stable/warnings.html
[   43s] =========================== short test summary info ============================
[   43s] FAILED tests/ipynb-test-samples/test-latex-pass-correctouput.ipynb::Cell 1
[   43s] ==================== 1 failed, 1 passed, 1 warning in 0.97s ====================

Testing test-latex-pass-correctouput.ipynb directky yields this:

abuild@skylab:~/rpmbuild/BUILD/nbval-0.9.6> PYTHONPATH=~/rpmbuild/BUILDROOT/python-nbval-0.9.6-0.x86_64/usr/lib/python3.8/site-packages pytest --nbval  -v tests/ipynb-test-samples/test-latex-pass-correctouput.ipynb --pdb
======================================================================================== test session starts =========================================================================================
platform linux -- Python 3.8.8, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3.8
cachedir: .pytest_cache
rootdir: /home/abuild/rpmbuild/BUILD/nbval-0.9.6
plugins: nbval-0.9.6, cov-2.11.1, timeout-1.4.2
collected 2 items                                                                                                                                                                                    

tests/ipynb-test-samples/test-latex-pass-correctouput::ipynb::Cell 0 PASSED                                                                                                                    [ 50%]
tests/ipynb-test-samples/test-latex-pass-correctouput::ipynb::Cell 1 FAILED                                                                                                                    [100%]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Notebook cell execution failed
Cell 1: Cell outputs differ

Input:
x**2 + y**3

Traceback:
Unexpected output fields from running code: {'stderr'}

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB post_mortem (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /home/abuild/rpmbuild/BUILDROOT/python-nbval-0.9.6-0.x86_64/usr/lib/python3.8/site-packages/nbval/plugin.py(550)raise_cell_error()
-> raise NbCellError(self.cell_num, message, self.cell.source, *args, **kwargs)
(Pdb) pp self.test_outputs
[{'name': 'stderr',
  'output_type': 'stream',
  'text': '/usr/lib/python3.8/site-packages/IPython/lib/latextools.py:126: '
          'MatplotlibDeprecationWarning: \n'
          'The to_png function was deprecated in Matplotlib 3.4 and will be '
          'removed two minor releases later. Use mathtext.math_to_image '
          'instead.\n'
          '  mt.to_png(f, s, fontsize=12, dpi=dpi, color=color)\n'
          '/usr/lib/python3.8/site-packages/IPython/lib/latextools.py:126: '
          'MatplotlibDeprecationWarning: \n'
          'The to_rgba function was deprecated in Matplotlib 3.4 and will be '
          'removed two minor releases later. Use mathtext.math_to_image '
          'instead.\n'
          '  mt.to_png(f, s, fontsize=12, dpi=dpi, color=color)\n'
          '/usr/lib/python3.8/site-packages/IPython/lib/latextools.py:126: '
          'MatplotlibDeprecationWarning: \n'
          'The to_mask function was deprecated in Matplotlib 3.4 and will be '
          'removed two minor releases later. Use mathtext.math_to_image '
          'instead.\n'
          '  mt.to_png(f, s, fontsize=12, dpi=dpi, color=color)\n'
          '/usr/lib/python3.8/site-packages/IPython/lib/latextools.py:126: '
          'MatplotlibDeprecationWarning: \n'
          'The MathtextBackendBitmap class was deprecated in Matplotlib 3.4 '
          'and will be removed two minor releases later. Use '
          'mathtext.math_to_image instead.\n'
          '  mt.to_png(f, s, fontsize=12, dpi=dpi, color=color)\n'},
 {'data': {'image/png': 'iVBORw0KGgoAAAANSUhEUgAAAEYAAAAaCAYAAAAKYioIAAADLUlEQVR4nO3YbYgVdRTH8c9uQoEbRK82KAsKLKhYTDFLIwzZMiJKfNGCD6FvzKAnzYdKFyRUilrRCoVgpXoTEUWCJloRPrxYq82KFMqEAiMDHxDt0XxxZmgY7129M7O3NfYLw9w5//+cOXPu//87//+0dHd3G+ZsWkvevwR9OI7D+AA3lg2qCczHXhH3cezGvdkOZRNzJ17FbZiMv7ANl5f0O9j8hEUYg7H4CO/h5rTDiJIP6Mxdz8Ax3C5Gz1Dl/dz1M5iHCWIklU5MnkvFKDxSsd/B5CJMRxt2pcaqE7MG/WLODnVuEnFeghN4AF+ljWU1JstLmIhp+LvA/bPxj9CtZrAfHRiP17BRpnDkE7NVBDctZ29Bb9K2qsZDXsZDQoAPlI+5YZ4UsT1Vp300fsenGdsf+A6fierajyfSxnxiFuI0Voi5l/IiZmEDFufuWePfpOw73zepmJ3J+dY67WvF+zw6gI9WXJy9yPIl3sANosLAUvGPvC2UO8sreBhdQnDbk6NtgAAGg89xSkyLPNMxRcS6N7GtwiRcI7RmpZjCb6U31dKY5/AblosMP48PRaJO5/o+IirRdhzKHAsae6/S/CkWmlfhiox9pNC+X7AsY2/Hm0JntmMc7sHmtEOtqvQjesSUWStK2INiTuZpKfQaHMTVddo+rmHbKMR5IHbiDrEWeTexLcOVYlQfy/Q9l6+65fpw5vccnDyXowbpwWU5WwfuF0k4mGvrPw+fqc6MF4m5Xojp7sRnQ9RKTJcQ25/FkHvM2dpSlp4attkiMb34pIDPXaIypQK8Tgju/MTeEHmNmZoE9rXYN+zHXFHuhjpH8C1uEX/uXViPL4o4yyZmIt4RG6xOMZ2eFaNqdfF4m8oOIbjr8avYAxUiTUwHNgmBmiIqC5GoPWKITyr6kCaS6kybWLQV3rO14jpsEfOwE9/n+ixJzi8UfUgT+SE59+H1Mo5GiGVx+wB9tilelhuhNznKkK7cCwlulio3kf81XbhPbAj7yjqr+rNDsxklEnItZuIbPF2F4ws9MXeLfc5R8VXucRUtRi/0xGxIjsr5P2lMpQwnpg7DianDGfIeoJNXJ4/1AAAAAElFTkSuQmCC\n',
           'text/latex': '$\\displaystyle x^{2} + y^{3}$',
           'text/plain': ' 2    3\nx  + y '},
  'execution_count': 2,
  'metadata': {},
  'output_type': 'execute_result'}]
(Pdb)
bnavigator commented 3 years ago

The deprecation warning is removed by https://github.com/ipython/ipython/pull/12889, but nbval probably should not trip by that with an older ipython.

bnavigator commented 3 years ago

Hacky workaround:

Index: nbval-0.9.6/tests/ipynb-test-samples/test-latex-pass-correctouput.ipynb
===================================================================
--- nbval-0.9.6.orig/tests/ipynb-test-samples/test-latex-pass-correctouput.ipynb
+++ nbval-0.9.6/tests/ipynb-test-samples/test-latex-pass-correctouput.ipynb
@@ -37,6 +37,9 @@
     }
    ],
    "source": [
+    "import warnings\n",
+    "from matplotlib import MatplotlibDeprecationWarning\n",
+    "warnings.filterwarnings(\"ignore\", category=MatplotlibDeprecationWarning)\n",
     "x**2 + y**3"
    ]
   }
Index: nbval-0.9.6/tests/latex-example.ipynb
===================================================================
--- nbval-0.9.6.orig/tests/latex-example.ipynb
+++ nbval-0.9.6/tests/latex-example.ipynb
@@ -15,6 +15,13 @@
    },
    "outputs": [],
    "source": [
+    "try:\n",
+    "    import warnings\n",
+    "    from matplotlib import MatplotlibDeprecationWarning\n",
+    "    warnings.filterwarnings(\"ignore\", category=MatplotlibDeprecationWarning)\n",
+    "except ImportError:\n",
+    "    # no matplotlib, no problem\n",
+    "    pass\n",
     "import sympy\n",
     "sympy.init_printing()\n",
     "x, y = sympy.symbols([\"x\", \"y\"])"