jupyter-xeus / xeus-zmq

ZeroMQ-based middleware for xeus
http://xeus-zmq.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
9 stars 12 forks source link

Tests fail, some of them intermittently #14

Open yurivict opened 1 year ago

yurivict commented 1 year ago
===>  Testing for xeus-zmq-1.0.2
-- xeus version: v1.0.2
-- xeus binary version: v1.0.0
-- XEUS_ZMQ_DISABLE_ARCH_NATIVE:        
-- XEUS_ZMQ_BUILD_SHARED_LIBS:          ON
-- XEUS_ZMQ_BUILD_STATIC_LIBS:          ON
-- XEUS_ZMQ_STATIC_DEPENDENCIES:        OFF
-- XEUS_ZMQ_BUILD_TESTS:                ON
-- CMAKE_CXX_FLAGS: -O2 -pipe -fstack-protector-strong -fno-strict-aliasing
-- CMAKE_CXX_FLAGS: -O2 -pipe -fstack-protector-strong -fno-strict-aliasing
-- tests enabled
-- Forcing tests build type to Release
Found xeus-zmq shared library.
-- Configuring done
-- Generating done
-- Build files have been written to: /usr/ports/devel/xeus-zmq/work/.build
ninja: no work to do.
[  0% 1/1] cd /usr/ports/devel/xeus-zmq/work/.build && /usr/local/bin/ctest --force-new-ctest-process
Test project /usr/ports/devel/xeus-zmq/work/.build
    Start 1: test_kernel
1/3 Test #1: test_kernel ......................***Failed    4.66 sec
============================= test session starts ==============================
platform freebsd13 -- Python 3.9.16, pytest-7.2.1, pluggy-1.0.0
Using --randomly-seed=3766723874
rootdir: /usr/ports/devel/xeus-zmq/work/.build/test
plugins: forked-1.4.0, hypothesis-6.68.2, mypy-plugins-1.10.1, cov-2.9.0, randomly-3.12.0, timeout-2.1.0, rerunfailures-10.1, flaky-3.7.0, xdist-2.5.0, env-0.6.2, freezegun-0.4.2, mock-3.10.0, flake8-1.1.1, typeguard-3.0.2
collected 13 items

test_kernel.py ss.s..ss....F                                             [100%]

=================================== FAILURES ===================================
_______________________ XeusKernelTests.test_xeus_stderr _______________________

self = <test_kernel.XeusKernelTests testMethod=test_xeus_stderr>

    def test_xeus_stderr(self):
        reply, output_msgs = self.execute_helper(code='error')
>       self.assertEqual(output_msgs[0]['msg_type'], 'stream')
E       IndexError: list index out of range

test_kernel.py:30: IndexError
=============================== warnings summary ===============================
../../../../../../local/lib/python3.9/site-packages/pytest_freezegun.py:17: 26 warnings
  /usr/local/lib/python3.9/site-packages/pytest_freezegun.py:17: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    if LooseVersion(pytest.__version__) < LooseVersion('3.6.0'):

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED test_kernel.py::XeusKernelTests::test_xeus_stderr - IndexError: list i...
============= 1 failed, 7 passed, 5 skipped, 26 warnings in 2.83s ==============

    Start 2: test_kernel_control
2/3 Test #2: test_kernel_control ..............***Failed    4.65 sec
============================= test session starts ==============================
platform freebsd13 -- Python 3.9.16, pytest-7.2.1, pluggy-1.0.0
Using --randomly-seed=223185949
rootdir: /usr/ports/devel/xeus-zmq/work/.build/test
plugins: forked-1.4.0, hypothesis-6.68.2, mypy-plugins-1.10.1, cov-2.9.0, randomly-3.12.0, timeout-2.1.0, rerunfailures-10.1, flaky-3.7.0, xdist-2.5.0, env-0.6.2, freezegun-0.4.2, mock-3.10.0, flake8-1.1.1, typeguard-3.0.2
collected 13 items

test_kernel_control.py .s.ss...s.s.F                                     [100%]

=================================== FAILURES ===================================
_______________________ XeusKernelTests.test_xeus_stderr _______________________

self = <test_kernel_control.XeusKernelTests testMethod=test_xeus_stderr>

    def test_xeus_stderr(self):
        reply, output_msgs = self.execute_helper(code='error')
>       self.assertEqual(output_msgs[0]['msg_type'], 'stream')
E       IndexError: list index out of range

test_kernel_control.py:30: IndexError
=============================== warnings summary ===============================
../../../../../../local/lib/python3.9/site-packages/pytest_freezegun.py:17: 26 warnings
  /usr/local/lib/python3.9/site-packages/pytest_freezegun.py:17: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    if LooseVersion(pytest.__version__) < LooseVersion('3.6.0'):

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED test_kernel_control.py::XeusKernelTests::test_xeus_stderr - IndexError...
============= 1 failed, 7 passed, 5 skipped, 26 warnings in 2.83s ==============

    Start 3: test_kernel_shell
3/3 Test #3: test_kernel_shell ................***Failed    4.66 sec
============================= test session starts ==============================
platform freebsd13 -- Python 3.9.16, pytest-7.2.1, pluggy-1.0.0
Using --randomly-seed=514591459
rootdir: /usr/ports/devel/xeus-zmq/work/.build/test
plugins: forked-1.4.0, hypothesis-6.68.2, mypy-plugins-1.10.1, cov-2.9.0, randomly-3.12.0, timeout-2.1.0, rerunfailures-10.1, flaky-3.7.0, xdist-2.5.0, env-0.6.2, freezegun-0.4.2, mock-3.10.0, flake8-1.1.1, typeguard-3.0.2
collected 13 items

test_kernel_shell.py ss..s.s....sF                                       [100%]

=================================== FAILURES ===================================
_______________________ XeusKernelTests.test_xeus_stderr _______________________

self = <test_kernel_shell.XeusKernelTests testMethod=test_xeus_stderr>

    def test_xeus_stderr(self):
        reply, output_msgs = self.execute_helper(code='error')
>       self.assertEqual(output_msgs[0]['msg_type'], 'stream')
E       IndexError: list index out of range

test_kernel_shell.py:30: IndexError
=============================== warnings summary ===============================
../../../../../../local/lib/python3.9/site-packages/pytest_freezegun.py:17: 26 warnings
  /usr/local/lib/python3.9/site-packages/pytest_freezegun.py:17: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    if LooseVersion(pytest.__version__) < LooseVersion('3.6.0'):

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED test_kernel_shell.py::XeusKernelTests::test_xeus_stderr - IndexError: ...
============= 1 failed, 7 passed, 5 skipped, 26 warnings in 2.85s ==============

0% tests passed, 3 tests failed out of 3

Total Test time (real) =  13.98 sec

The following tests FAILED:
      1 - test_kernel (Failed)
      2 - test_kernel_control (Failed)
      3 - test_kernel_shell (Failed)
Errors while running CTest
FAILED: CMakeFiles/test.util 
cd /usr/ports/devel/xeus-zmq/work/.build && /usr/local/bin/ctest --force-new-ctest-process
ninja: build stopped: subcommand failed.
*** Error code 1

Stop.
make: stopped in /usr/ports/devel/xeus-zmq
[yuri@yv /usr/ports/devel/xeus-zmq]$ ^C
[yuri@yv /usr/ports/devel/xeus-zmq]$ 
[yuri@yv /usr/ports/devel/xeus-zmq]$ 
[yuri@yv /usr/ports/devel/xeus-zmq]$ CTEST_OUTPUT_ON_FAILURE=1 make test
===>  Testing for xeus-zmq-1.0.2
-- xeus version: v1.0.2
-- xeus binary version: v1.0.0
-- XEUS_ZMQ_DISABLE_ARCH_NATIVE:        
-- XEUS_ZMQ_BUILD_SHARED_LIBS:          ON
-- XEUS_ZMQ_BUILD_STATIC_LIBS:          ON
-- XEUS_ZMQ_STATIC_DEPENDENCIES:        OFF
-- XEUS_ZMQ_BUILD_TESTS:                ON
-- CMAKE_CXX_FLAGS: -O2 -pipe -fstack-protector-strong -fno-strict-aliasing
-- CMAKE_CXX_FLAGS: -O2 -pipe -fstack-protector-strong -fno-strict-aliasing
-- tests enabled
-- Forcing tests build type to Release
Found xeus-zmq shared library.
-- Configuring done
-- Generating done
-- Build files have been written to: /usr/ports/devel/xeus-zmq/work/.build
ninja: no work to do.
[  0% 1/1] cd /usr/ports/devel/xeus-zmq/work/.build && /usr/local/bin/ctest --force-new-ctest-process
Test project /usr/ports/devel/xeus-zmq/work/.build
    Start 1: test_kernel
1/3 Test #1: test_kernel ......................***Failed    4.66 sec
============================= test session starts ==============================
platform freebsd13 -- Python 3.9.16, pytest-7.2.1, pluggy-1.0.0
Using --randomly-seed=2865058251
rootdir: /usr/ports/devel/xeus-zmq/work/.build/test
plugins: forked-1.4.0, hypothesis-6.68.2, mypy-plugins-1.10.1, cov-2.9.0, randomly-3.12.0, timeout-2.1.0, rerunfailures-10.1, flaky-3.7.0, xdist-2.5.0, env-0.6.2, freezegun-0.4.2, mock-3.10.0, flake8-1.1.1, typeguard-3.0.2
collected 13 items

test_kernel.py s.F.sss.s....                                             [100%]

=================================== FAILURES ===================================
_______________________ XeusKernelTests.test_xeus_stderr _______________________

self = <test_kernel.XeusKernelTests testMethod=test_xeus_stderr>

    def test_xeus_stderr(self):
        reply, output_msgs = self.execute_helper(code='error')
>       self.assertEqual(output_msgs[0]['msg_type'], 'stream')
E       IndexError: list index out of range

test_kernel.py:30: IndexError
=============================== warnings summary ===============================
../../../../../../local/lib/python3.9/site-packages/pytest_freezegun.py:17: 26 warnings
  /usr/local/lib/python3.9/site-packages/pytest_freezegun.py:17: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    if LooseVersion(pytest.__version__) < LooseVersion('3.6.0'):

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED test_kernel.py::XeusKernelTests::test_xeus_stderr - IndexError: list i...
============= 1 failed, 7 passed, 5 skipped, 26 warnings in 2.84s ==============

    Start 2: test_kernel_control
2/3 Test #2: test_kernel_control ..............   Passed    4.51 sec
    Start 3: test_kernel_shell
3/3 Test #3: test_kernel_shell ................***Failed    4.68 sec
============================= test session starts ==============================
platform freebsd13 -- Python 3.9.16, pytest-7.2.1, pluggy-1.0.0
Using --randomly-seed=4048850708
rootdir: /usr/ports/devel/xeus-zmq/work/.build/test
plugins: forked-1.4.0, hypothesis-6.68.2, mypy-plugins-1.10.1, cov-2.9.0, randomly-3.12.0, timeout-2.1.0, rerunfailures-10.1, flaky-3.7.0, xdist-2.5.0, env-0.6.2, freezegun-0.4.2, mock-3.10.0, flake8-1.1.1, typeguard-3.0.2
collected 13 items

test_kernel_shell.py s.Fs.s....s.s                                       [100%]

=================================== FAILURES ===================================
_______________________ XeusKernelTests.test_xeus_stderr _______________________

self = <test_kernel_shell.XeusKernelTests testMethod=test_xeus_stderr>

    def test_xeus_stderr(self):
        reply, output_msgs = self.execute_helper(code='error')
>       self.assertEqual(output_msgs[0]['msg_type'], 'stream')
E       IndexError: list index out of range

test_kernel_shell.py:30: IndexError
=============================== warnings summary ===============================
../../../../../../local/lib/python3.9/site-packages/pytest_freezegun.py:17: 26 warnings
  /usr/local/lib/python3.9/site-packages/pytest_freezegun.py:17: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    if LooseVersion(pytest.__version__) < LooseVersion('3.6.0'):

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED test_kernel_shell.py::XeusKernelTests::test_xeus_stderr - IndexError: ...
============= 1 failed, 7 passed, 5 skipped, 26 warnings in 2.85s ==============

33% tests passed, 2 tests failed out of 3

Total Test time (real) =  13.85 sec

The following tests FAILED:
      1 - test_kernel (Failed)
      3 - test_kernel_shell (Failed)
Errors while running CTest
FAILED: CMakeFiles/test.util 
cd /usr/ports/devel/xeus-zmq/work/.build && /usr/local/bin/ctest --force-new-ctest-process
ninja: build stopped: subcommand failed.
*** Error code 1

Stop.
make: stopped in /usr/ports/devel/xeus-zmq
[yuri@yv /usr/ports/devel/xeus-zmq]$ CTEST_OUTPUT_ON_FAILURE=1 make test
===>  Testing for xeus-zmq-1.0.2
-- xeus version: v1.0.2
-- xeus binary version: v1.0.0
-- XEUS_ZMQ_DISABLE_ARCH_NATIVE:        
-- XEUS_ZMQ_BUILD_SHARED_LIBS:          ON
-- XEUS_ZMQ_BUILD_STATIC_LIBS:          ON
-- XEUS_ZMQ_STATIC_DEPENDENCIES:        OFF
-- XEUS_ZMQ_BUILD_TESTS:                ON
-- CMAKE_CXX_FLAGS: -O2 -pipe -fstack-protector-strong -fno-strict-aliasing
-- CMAKE_CXX_FLAGS: -O2 -pipe -fstack-protector-strong -fno-strict-aliasing
-- tests enabled
-- Forcing tests build type to Release
Found xeus-zmq shared library.
-- Configuring done
-- Generating done
-- Build files have been written to: /usr/ports/devel/xeus-zmq/work/.build
ninja: no work to do.
[  0% 1/1] cd /usr/ports/devel/xeus-zmq/work/.build && /usr/local/bin/ctest --force-new-ctest-process
Test project /usr/ports/devel/xeus-zmq/work/.build
    Start 1: test_kernel
1/3 Test #1: test_kernel ......................   Passed    4.48 sec
    Start 2: test_kernel_control
2/3 Test #2: test_kernel_control ..............***Failed    4.68 sec
============================= test session starts ==============================
platform freebsd13 -- Python 3.9.16, pytest-7.2.1, pluggy-1.0.0
Using --randomly-seed=2171778117
rootdir: /usr/ports/devel/xeus-zmq/work/.build/test
plugins: forked-1.4.0, hypothesis-6.68.2, mypy-plugins-1.10.1, cov-2.9.0, randomly-3.12.0, timeout-2.1.0, rerunfailures-10.1, flaky-3.7.0, xdist-2.5.0, env-0.6.2, freezegun-0.4.2, mock-3.10.0, flake8-1.1.1, typeguard-3.0.2
collected 13 items

test_kernel_control.py ss..ss..sF...                                     [100%]

=================================== FAILURES ===================================
_______________________ XeusKernelTests.test_xeus_stderr _______________________

self = <test_kernel_control.XeusKernelTests testMethod=test_xeus_stderr>

    def test_xeus_stderr(self):
        reply, output_msgs = self.execute_helper(code='error')
>       self.assertEqual(output_msgs[0]['msg_type'], 'stream')
E       IndexError: list index out of range

test_kernel_control.py:30: IndexError
=============================== warnings summary ===============================
../../../../../../local/lib/python3.9/site-packages/pytest_freezegun.py:17: 26 warnings
  /usr/local/lib/python3.9/site-packages/pytest_freezegun.py:17: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    if LooseVersion(pytest.__version__) < LooseVersion('3.6.0'):

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED test_kernel_control.py::XeusKernelTests::test_xeus_stderr - IndexError...
============= 1 failed, 7 passed, 5 skipped, 26 warnings in 2.85s ==============

    Start 3: test_kernel_shell
3/3 Test #3: test_kernel_shell ................***Failed    4.72 sec
============================= test session starts ==============================
platform freebsd13 -- Python 3.9.16, pytest-7.2.1, pluggy-1.0.0
Using --randomly-seed=1569082332
rootdir: /usr/ports/devel/xeus-zmq/work/.build/test
plugins: forked-1.4.0, hypothesis-6.68.2, mypy-plugins-1.10.1, cov-2.9.0, randomly-3.12.0, timeout-2.1.0, rerunfailures-10.1, flaky-3.7.0, xdist-2.5.0, env-0.6.2, freezegun-0.4.2, mock-3.10.0, flake8-1.1.1, typeguard-3.0.2
collected 13 items

test_kernel_shell.py .....s.Fsss.s                                       [100%]

=================================== FAILURES ===================================
_______________________ XeusKernelTests.test_xeus_stderr _______________________

self = <test_kernel_shell.XeusKernelTests testMethod=test_xeus_stderr>

    def test_xeus_stderr(self):
        reply, output_msgs = self.execute_helper(code='error')
>       self.assertEqual(output_msgs[0]['msg_type'], 'stream')
E       IndexError: list index out of range

test_kernel_shell.py:30: IndexError
=============================== warnings summary ===============================
../../../../../../local/lib/python3.9/site-packages/pytest_freezegun.py:17: 26 warnings
  /usr/local/lib/python3.9/site-packages/pytest_freezegun.py:17: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    if LooseVersion(pytest.__version__) < LooseVersion('3.6.0'):

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED test_kernel_shell.py::XeusKernelTests::test_xeus_stderr - IndexError: ...
============= 1 failed, 7 passed, 5 skipped, 26 warnings in 2.83s ==============

33% tests passed, 2 tests failed out of 3

Total Test time (real) =  13.88 sec

The following tests FAILED:
      2 - test_kernel_control (Failed)
      3 - test_kernel_shell (Failed)
Errors while running CTest
FAILED: CMakeFiles/test.util 
cd /usr/ports/devel/xeus-zmq/work/.build && /usr/local/bin/ctest --force-new-ctest-process
ninja: build stopped: subcommand failed.

Version: 1.0.2 jupyter-kernel-test-0.5.0 Python-3.9 FreeBSD 13.1