Open kloczek opened 2 years ago
Hi @kloczek , thank you for the report!
First I have a question for you - would you be interested in automating the build of the RPM package? Do you think we could do this on our CI with an appropriate GitHub action? Hopefully that could spare you some time and also I guess I would break less often the RPM packaging process :smile:
Regarding the test failures, I think the current conditions for skipping some tests are not appropriate for your environment. In particular I see these two errors:
test_mirror.py::test_ipynb_to_pandoc
: I'd say that we are not using the same version of pandoc
, so the condition for enabling these tests (is_pandoc_available
) should be improved.Hi @kloczek , I have tried to fix a few of the failures in this branch - can you tell me if you still see issues with that branch?
Regarding the version of jupyter_packaging
I am afraid that changing the version might have side effects - I'll have a look later on.
First I have a question for you - would you be interested in automating the build of the RPM package? Do you think we could do this on our CI with an appropriate GitHub action? Hopefully that could spare you some time and also I guess I would break less often the RPM packaging process 😄
I have already such automation 😄 IMO building in CI rpm package does not make to much sense because details of the build, install and testing procedure may depend on OS distribution which are using usiually different versions of the python and exaxct moodules.
Hi @kloczek , I have tried to fix a few of the failures in this branch - can you tell me if you still see issues with that branch?
WIll check that ASP. Thank you 😄
I have tried to fix a few of the failures in this branch
FYI I have merged this into main
already, as they happened to fix some issues I was also having locally.
MO building in CI rpm package does not make to much sense because details of the build, install and testing procedure may depend on OS distribution which are using usually different versions of the python and exact modules.
Well maybe it is already a good thing if this work on the Linux distribution made available on GitHub actions?
Just tested 1.13.7. I still must use my patch to relax build dependencies however looks like some progress has been made because now are failing only four units
looks like some progress has been made because now are failing only four units
Ha ha thank you @kloczek! And I see that three of them are tests that I just added in the latest release... I really need to add an entry on the CI that will test the package in the absence of a kernel.
So should I add ipkernel
(?) to my build depenedencies or temporary hold such change? 😄
So should I add ipkernel (?) to my build depenedencies or temporary hold such change? smile
Well you don't need to (I prefer to fix the test requirements and make sure we skip the tests that cannot succeed). But if you want to, that will increase the coverage. What you need to do is to make sure ipykernel
is available and then create a kernel with e.g. this command:
https://github.com/mwouts/jupytext/blob/8cfa66bef380bf6d54479a06b092da30dbc9a7c4/.github/workflows/continuous-integration.yml#L99
By the way I have integrated on main
the dynamic skip for the four tests that were failing in the absence of a kernel, let me know if the RPM package passes with this change.
Looks like something has changes iin 1.14.2 and now pytest is failing in new way
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-jupytext-1.14.2-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-jupytext-1.14.2-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra --deselect tests/test_contentsmanager.py::test_python_kernel_preserves_R_files --deselect tests/test_execute.py::test_cat_execute_does_not_update_the_metadata --deselect tests/test_execute.py::test_execute_text_file_does_update_the_metadata --deselect tests/test_using_cli.py::test_jupytext_commands_in_the_documentation_work
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.15, pytest-7.2.0, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/jupytext-1.14.2, configfile: pytest.ini
plugins: anyio-3.6.2, black-0.3.12
collected 2839 items / 4 deselected / 2835 selected
tests/test_active_cells.py ............................ [ 0%]
tests/test_auto_ext.py .................................................................x.s........................ [ 4%]
tests/test_black.py ...sss....... [ 4%]
tests/test_cell_id.py . [ 4%]
tests/test_cell_markers.py .. [ 4%]
tests/test_cell_metadata.py ............................................................ [ 6%]
tests/test_cells.py ................ [ 7%]
tests/test_changelog.py ... [ 7%]
tests/test_cli.py ................................................^C
[tkloczko@pers-jacek SPECS]$
============================================================================= warnings summary =============================================================================
jupytext/cell_reader.py:20
/home/tkloczko/rpmbuild/BUILD/jupytext-1.14.2/jupytext/cell_reader.py:20: UserWarning: Sphinx Gallery in version 0.11.1 is not supported by Jupytext. Please use sphinx-gallery<=0.7.0 instead. If that is an issue, feel free to report it at https://github.com/mwouts/jupytext/issues, or even better, prepare a PR to handle the new signature of sphinx_gallery.notebook.rst2md.
warnings.warn(
../../../../../usr/lib/python3.8/site-packages/notebook/nbextensions.py:15
/usr/lib/python3.8/site-packages/notebook/nbextensions.py:15: DeprecationWarning: Jupyter is migrating its paths to use standard platformdirs
given by the platformdirs library. To remove this warning and
see the appropriate new directories, set the environment variable
`JUPYTER_PLATFORM_DIRS=1` and then run `jupyter --paths`.
The use of platformdirs will be the default in `jupyter_core` v6
from jupyter_core.paths import (
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================================================= short test summary info ==========================================================================
SKIPPED [1] tests/test_auto_ext.py:36: No file_extension in language_info
SKIPPED [1] tests/test_black.py:68: autopep8 not found
SKIPPED [1] tests/test_black.py:79: flake8 not found
SKIPPED [1] tests/test_black.py:91: flake8 not found
XFAIL tests/test_auto_ext.py::test_auto_from_kernelspecs_works[/home/tkloczko/rpmbuild/BUILD/jupytext-1.14.2/tests/notebooks/ipynb_sage/sage_print_hello.ipynb] - reason: Sage notebooks have Python in their language_info metadata, see #727
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
/home/tkloczko/rpmbuild/BUILD/jupytext-1.14.2/jupytext/magics.py:118: KeyboardInterrupt
(to show a full traceback on KeyboardInterrupt use --full-trace)
=================================================== 258 passed, 4 skipped, 4 deselected, 1 xfailed, 2 warnings in 9.35s ====================================================
Hello @kloczek, sorry about that! Just to confirm, is it the xfailed
test that causes an issue? Would you prefer me to skip it?
I really have no idea what test suite is does so giving you advices wouls be not proper 😋
Does jupytext==1.14.3
or jupytext==1.14.4
works better for you? Just in case I have replaced the xfailed
with a skip
. Cheers
Yes it is (+/- one waring)
============================================================================= warnings summary =============================================================================
jupytext/cell_reader.py:20
/home/tkloczko/rpmbuild/BUILD/jupytext-1.14.4/jupytext/cell_reader.py:20: UserWarning: Sphinx Gallery in version 0.11.1 is not supported by Jupytext. Please use sphinx-gallery<=0.7.0 instead. If that is an issue, feel free to report it at https://github.com/mwouts/jupytext/issues, or even better, prepare a PR to handle the new signature of sphinx_gallery.notebook.rst2md.
warnings.warn(
../../../../../usr/lib/python3.8/site-packages/notebook/nbextensions.py:15
/usr/lib/python3.8/site-packages/notebook/nbextensions.py:15: DeprecationWarning: Jupyter is migrating its paths to use standard platformdirs
given by the platformdirs library. To remove this warning and
see the appropriate new directories, set the environment variable
`JUPYTER_PLATFORM_DIRS=1` and then run `jupyter --paths`.
The use of platformdirs will be the default in `jupyter_core` v6
from jupyter_core.paths import (
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================================================= short test summary info ==========================================================================
SKIPPED [1] tests/test_auto_ext.py:43: Sage notebooks have Python in their language_info metadata, see #727
SKIPPED [1] tests/test_auto_ext.py:36: No file_extension in language_info
SKIPPED [1] tests/test_black.py:68: autopep8 not found
SKIPPED [1] tests/test_black.py:79: flake8 not found
SKIPPED [1] tests/test_black.py:91: flake8 not found
SKIPPED [2] tests/test_cli.py:439: Please run 'python -m ipykernel install --name python_kernel --user'
SKIPPED [2] tests/test_cli.py:454: Please run 'python -m ipykernel install --name python_kernel --user'
SKIPPED [2] tests/test_cli.py:470: Please run 'python -m ipykernel install --name python_kernel --user'
SKIPPED [15] tests/test_cli.py:559: pandoc>=2.11 is not available
SKIPPED [1] tests/test_cli.py:676: Please run 'python -m ipykernel install --name python_kernel --user'
SKIPPED [1] tests/test_cli.py:691: Please run 'python -m ipykernel install --name python_kernel --user'
SKIPPED [1] tests/test_cli.py:841: sphinx_gallery is not available
SKIPPED [1] tests/test_cli.py:1051: pandoc>=2.11 is not available
SKIPPED [5] tests/test_cli.py:1068: Please run 'python -m ipykernel install --name python_kernel --user'
SKIPPED [1] tests/test_cli.py:1139: pandoc>=2.11 is not available
SKIPPED [1] tests/test_cli.py:1259: Please run 'python -m ipykernel install --name python_kernel --user'
SKIPPED [17] tests/test_contentsmanager.py:180: pandoc>=2.11 is not available
SKIPPED [7] tests/test_contentsmanager.py:203: quarto>=0.2 is not available
SKIPPED [1] tests/test_contentsmanager.py:1219: Please run 'python -m ipykernel install --name python_kernel --user'
SKIPPED [1] tests/test_contentsmanager.py:1290: sphinx_gallery is not available
SKIPPED [2] tests/test_contentsmanager.py:1867: jupyterfs is not available
SKIPPED [1] tests/test_execute.py:18: Please run 'python -m ipykernel install --name python_kernel --user'
SKIPPED [1] tests/test_execute.py:48: Please run 'python -m ipykernel install --name python_kernel --user'
SKIPPED [1] tests/test_execute.py:79: Please run 'python -m ipykernel install --name python_kernel --user'
SKIPPED [1] tests/test_execute.py:99: Please run 'python -m ipykernel install --name python_kernel --user'
SKIPPED [1] tests/test_execute.py:118: Please run 'python -m ipykernel install --name python_kernel --user'
SKIPPED [1] tests/test_execute.py:143: Please run 'python -m ipykernel install --name python_kernel --user'
SKIPPED [1] tests/test_execute.py:163: irkernel is not installed
SKIPPED [1] tests/test_execute.py:185: Please run 'python -m ipykernel install --name python_kernel --user'
SKIPPED [1] tests/test_execute.py:266: Please run 'python -m ipykernel install --name python_kernel --user'
SKIPPED [1] tests/test_execute.py:278: Please run 'python -m ipykernel install --name python_kernel --user'
SKIPPED [1] tests/test_formats.py:350: pandoc>=2.11 is not available
SKIPPED [1] tests/test_ipynb_to_myst.py:162: myst is available
SKIPPED [1] tests/test_ipynb_to_myst.py:171: myst is available
SKIPPED [1] tests/test_isort.py:8: isort not found
SKIPPED [17] tests/test_mirror.py:132: pandoc>=2.11 is not available
SKIPPED [7] tests/test_mirror.py:141: quarto>=0.2 is not available
SKIPPED [11] tests/test_mirror.py:168: sphinx_gallery is not available
SKIPPED [1] tests/test_mirror.py:219: sphinx_gallery is not available
SKIPPED [1] tests/test_mirror.py:225: sphinx_gallery is not available
SKIPPED [1] tests/test_pre_commit_0_ipynb_to_py.py:16: Jupytext folder is not a git repository #814
SKIPPED [1] tests/test_pre_commit_1_sync.py:17: Jupytext folder is not a git repository #814
SKIPPED [1] tests/test_pre_commit_1_sync_with_config.py:14: Jupytext folder is not a git repository #814
SKIPPED [1] tests/test_pre_commit_1_sync_with_no_config.py:18: Jupytext folder is not a git repository #814
SKIPPED [1] tests/test_pre_commit_2_sync_nbstripout.py:14: Jupytext folder is not a git repository #814
SKIPPED [1] tests/test_pre_commit_3_sync_black_nbstripout.py:13: Jupytext folder is not a git repository #814
SKIPPED [1] tests/test_pre_commit_4_sync_execute.py:15: Jupytext folder is not a git repository #814
SKIPPED [1] tests/test_pre_commit_5_reformat_markdown.py:15: Jupytext folder is not a git repository #814
SKIPPED [1] tests/test_pre_commit_scripts.py:217: flake8 not found
SKIPPED [1] tests/test_pre_commit_scripts.py:334: pandoc>=2.11 is not available
SKIPPED [1] tests/test_read_empty_text_notebook.py:14: quarto is not available
SKIPPED [1] tests/test_read_simple_pandoc.py:12: pandoc>=2.11 is not available
SKIPPED [1] tests/test_read_simple_pandoc.py:34: pandoc>=2.11 is not available
SKIPPED [1] tests/test_read_simple_pandoc.py:49: pandoc>=2.11 is not available
SKIPPED [1] tests/test_read_simple_pandoc.py:63: pandoc>=2.11 is not available
SKIPPED [1] tests/test_read_simple_pandoc.py:81: Pandoc is installed
SKIPPED [1] tests/test_read_simple_quarto.py:9: quarto>=0.2 is not available
SKIPPED [1] tests/test_using_cli.py:15: Please run 'python -m ipykernel install --name python_kernel --user'
========================================================= 2716 passed, 134 skipped, 2 warnings in 88.27s (0:01:28) =========================================================
Feel free to close this ticket.
I'm trying to package your module as an rpm package. So I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.
python3 -sBm build -w --no-isolation
build
with--no-isolation
I'm using during all processes only locally installed modulesBecause I'm trying to build all my packages only agains what is installed (
--no-isolation
) and I havejupyter-packaging
0.11.1 I've added the patch:I'm not sure is that caused that pytest is failing. Latest pytets shows at the end few minor warnings. Here is pytest output: