0.5.0: pytest is failing in two units

[kloczek]

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix>
• run pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

Here is pytest output:

```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-events-0.5.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-events-0.5.0-2.fc35.x86_64/usr/lib/python3.8/site-packages + /usr/bin/pytest -ra =========================================================================== test session starts ============================================================================ platform linux -- Python 3.8.15, pytest-7.2.0, pluggy-1.0.0 rootdir: /home/tkloczko/rpmbuild/BUILD/jupyter_events-0.5.0, configfile: pyproject.toml, testpaths: tests/ plugins: anyio-3.6.2, console-scripts-1.3.1, asyncio-0.20.2 asyncio: mode=auto collected 49 items tests/test_cli.py ....... [ 14%] tests/test_listeners.py ..... [ 24%] tests/test_logger.py .........F... [ 51%] tests/test_modifiers.py ...... [ 63%] tests/test_schema.py ..F............ [ 93%] tests/test_traits.py ... [100%] ================================================================================= FAILURES ================================================================================= ________________________________________________________________________ test_emit_badschema_format ________________________________________________________________________ def test_emit_badschema_format(): """ Fail fast when an event doesn't conform to a specific format """ schema = { "$id": "http://test/test", "version": 1, "type": "object", "properties": { "something": {"type": "string", "title": "test", "format": "date-time"}, }, } el = EventLogger(handlers=[logging.NullHandler()]) el.register_event_schema(schema) with pytest.raises(jsonschema.ValidationError) as excinfo: > el.emit(schema_id="http://test/test", data={"something": "chucknorris"}) E Failed: DID NOT RAISE tests/test_logger.py:264: Failed _______________________________________________________ test_bad_validations[bad-id.yaml-'not-a-uri' is not a 'uri'] _______________________________________________________ schema_file = 'bad-id.yaml', validation_error_msg = "'not-a-uri' is not a 'uri'" @pytest.mark.parametrize("schema_file,validation_error_msg", BAD_SCHEMAS) def test_bad_validations(schema_file, validation_error_msg): """ Validation fails because the schema is missing a redactionPolicies field. """ # Read the schema file with open(SCHEMA_PATH / "bad" / schema_file) as f: schema = yaml.loads(f) # Assert that the schema files for a known reason. with pytest.raises(ValidationError) as err: > validate_schema(schema) E Failed: DID NOT RAISE tests/test_schema.py:38: Failed =========================================================================== slowest 10 durations =========================================================================== 0.27s call tests/test_cli.py::test_cli_version[subprocess] 0.27s call tests/test_cli.py::test_cli_invalid[subprocess] 0.27s call tests/test_cli.py::test_cli_good_raw[subprocess] 0.27s call tests/test_cli.py::test_cli_good[subprocess] 0.26s call tests/test_cli.py::test_cli_missing[subprocess] 0.26s call tests/test_cli.py::test_cli_malformed[subprocess] 0.26s call tests/test_cli.py::test_cli_help[subprocess] 0.01s setup tests/test_listeners.py::test_listener_function 0.01s teardown tests/test_modifiers.py::test_modifier_function (1 durations < 0.005s hidden. Use -vv to show these durations.) ========================================================================= short test summary info ========================================================================== FAILED tests/test_logger.py::test_emit_badschema_format - Failed: DID NOT RAISE FAILED tests/test_schema.py::test_bad_validations[bad-id.yaml-'not-a-uri' is not a 'uri'] - Failed: DID NOT RAISE ======================================================================= 2 failed, 47 passed in 2.28s ======================================================================= ```

Here is list of installed modules in build env

```console Package Version ----------------------------- ----------------- alabaster 0.7.12 anyio 3.6.2 appdirs 1.4.4 argon2-cffi 21.3.0 argon2-cffi-bindings 21.2.0 attrs 22.1.0 Babel 2.11.0 beautifulsoup4 4.11.1 bleach 5.0.0 Brlapi 0.8.3 build 0.9.0 cffi 1.15.1 charset-normalizer 3.0.1 click 8.1.3 cloudpickle 2.2.0 commonmark 0.9.1 contourpy 1.0.6 cssselect 1.1.0 cycler 0.11.0 defusedxml 0.7.1 distro 1.8.0 dnspython 2.2.1 docutils 0.19 doit 0.36.0 editables 0.3 entrypoints 0.4 exceptiongroup 1.0.0 extras 1.0.0 fastjsonschema 2.16.1 fixtures 4.0.0 fonttools 4.38.0 gpg 1.17.1-unknown hatchling 1.11.1 html5lib 1.1 idna 3.4 imagesize 1.4.1 importlib-metadata 5.1.0 importlib-resources 5.9.0 iniconfig 1.1.1 Jinja2 3.1.2 json5 0.9.9 jsonschema 4.17.3 jupyter_client 7.4.8 jupyter_core 5.1.0 jupyter-server 1.23.3 jupyterlab-pygments 0.1.2 jupyterlab_server 2.16.3 kiwisolver 1.4.4 libcomps 0.1.19 louis 3.23.0 lxml 4.9.1 markdown-it-py 2.1.0 MarkupSafe 2.1.1 matplotlib 3.6.2 mdit-py-plugins 0.3.3 mdurl 0.1.2 mistune 2.0.4 myst-parser 0.18.1 nbclient 0.7.2 nbconvert 7.2.6 nbformat 5.7.0 nest-asyncio 1.5.6 numpy 1.23.1 olefile 0.46 packaging 21.3 pandocfilters 1.5.0 pathspec 0.10.2 pbr 5.9.0 pep517 0.13.0 Pillow 9.3.0 pip 22.3.1 pkgutil_resolve_name 1.3.10 platformdirs 2.5.2 pluggy 1.0.0 ply 3.11 prometheus-client 0.15.0 ptyprocess 0.7.0 pycparser 2.21 Pygments 2.13.0 PyGObject 3.42.2 pyparsing 3.0.9 pyrsistent 0.19.2 pytest 7.2.0 pytest-asyncio 0.20.2 pytest-console-scripts 1.3.1 python-dateutil 2.8.2 python-json-logger 2.0.4 pytz 2022.4 PyYAML 6.0 pyzmq 24.0.0 requests 2.28.1 rich 12.6.0 rpm 4.17.0 scour 0.38.2 Send2Trash 1.8.0 setuptools 65.6.3 six 1.16.0 sniffio 1.2.0 snowballstemmer 2.2.0 soupsieve 2.3.2.post1 Sphinx 5.3.0 sphinxcontrib-applehelp 1.0.2.dev20221204 sphinxcontrib-devhelp 1.0.2.dev20221204 sphinxcontrib-htmlhelp 2.0.0 sphinxcontrib-jsmath 1.0.1.dev20221204 sphinxcontrib-qthelp 1.0.3.dev20221204 sphinxcontrib-serializinghtml 1.1.5 terminado 0.15.0 testtools 2.5.0 tinycss2 1.2.1 tomli 2.0.1 tornado 6.2 traitlets 5.4.0 typing_extensions 4.4.0 urllib3 1.26.12 webencodings 0.5.1 websocket-client 1.4.2 wheel 0.38.4 zipp 3.11.0 ```

[bretttully]

I could be wrong, but I believe this is because this library has introduced a dependency on jsonschema>=4.3 but this is not pinned. There is a similar but different bug report over on the conda-forge feedstock which has made Jupyterlab unusable.

[bretttully]

See https://github.com/conda-forge/jupyter_events-feedstock/issues/5

[blink1073]

The source package has the minimum bound, but not yet the conda package. I'll update the conda recipe.

[bretttully]

Sorry it's early here, I looked at that and completely missed it. 🤦‍♂️

[kevin-bates]

I believe this should be resolved by #59 since the test failures imply (and the package list confirms) that the additional validators are not installed.

[kevin-bates]

Hi @kloczek - could you please retry the rpm package build such that it includes jupyter_events >= 0.6.2? I suspect this issue should be resolved. If not resolved, please provide the current output information you included in the original description.

[kloczek]

I just tested 0.6.3 and unit still are failing

```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-events-0.6.3-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-events-0.6.3-2.fc35.x86_64/usr/lib/python3.8/site-packages + /usr/bin/pytest -ra -m 'not network' ==================================================================================== test session starts ==================================================================================== platform linux -- Python 3.8.16, pytest-7.2.1, pluggy-1.0.0 rootdir: /home/tkloczko/rpmbuild/BUILD/jupyter_events-0.6.3, configfile: pyproject.toml, testpaths: tests/ plugins: anyio-3.6.2, asyncio-0.20.3, console-scripts-1.3.1 asyncio: mode=auto collected 49 items tests/test_cli.py ....... [ 14%] tests/test_listeners.py ..... [ 24%] tests/test_logger.py .........F... [ 51%] tests/test_modifiers.py ...... [ 63%] tests/test_schema.py ..F............ [ 93%] tests/test_traits.py ... [100%] ========================================================================================= FAILURES ========================================================================================== ________________________________________________________________________________ test_emit_badschema_format _________________________________________________________________________________ def test_emit_badschema_format(): """ Fail fast when an event doesn't conform to a specific format """ schema = { "$id": "http://test/test", "version": 1, "type": "object", "properties": { "something": {"type": "string", "title": "test", "format": "date-time"}, }, } el = EventLogger(handlers=[logging.NullHandler()]) el.register_event_schema(schema) with pytest.raises(jsonschema.ValidationError) as excinfo: > el.emit(schema_id="http://test/test", data={"something": "chucknorris"}) E Failed: DID NOT RAISE tests/test_logger.py:262: Failed _______________________________________________________________ test_bad_validations[bad-id.yaml-'not-a-uri' is not a 'uri'] ________________________________________________________________ schema_file = 'bad-id.yaml', validation_error_msg = "'not-a-uri' is not a 'uri'" @pytest.mark.parametrize("schema_file,validation_error_msg", BAD_SCHEMAS) def test_bad_validations(schema_file, validation_error_msg): """ Validation fails because the schema is missing a redactionPolicies field. """ # Read the schema file with open(SCHEMA_PATH / "bad" / schema_file) as f: schema = yaml.loads(f) # Assert that the schema files for a known reason. with pytest.raises(ValidationError) as err: > validate_schema(schema) E Failed: DID NOT RAISE tests/test_schema.py:38: Failed =================================================================================== slowest 10 durations ==================================================================================== 0.27s call tests/test_cli.py::test_cli_version[subprocess] 0.27s call tests/test_cli.py::test_cli_invalid[subprocess] 0.26s call tests/test_cli.py::test_cli_good_raw[subprocess] 0.26s call tests/test_cli.py::test_cli_good[subprocess] 0.25s call tests/test_cli.py::test_cli_missing[subprocess] 0.25s call tests/test_cli.py::test_cli_malformed[subprocess] 0.25s call tests/test_cli.py::test_cli_help[subprocess] 0.01s setup tests/test_listeners.py::test_listener_function 0.01s setup tests/test_listeners.py::test_bad_listener_function_signature 0.01s setup tests/test_listeners.py::test_remove_listener_function ================================================================================== short test summary info ================================================================================== FAILED tests/test_logger.py::test_emit_badschema_format - Failed: DID NOT RAISE FAILED tests/test_schema.py::test_bad_validations[bad-id.yaml-'not-a-uri' is not a 'uri'] - Failed: DID NOT RAISE =============================================================================== 2 failed, 47 passed in 2.27s ================================================================================ ```

[kevin-bates]

This implies that neither of the validators are installed (rfc3339-validator and rfc3986-validator) that are supposed to come with the jsonschema[format-nongpl] dependency and which we added as explicit dependencies in 0.6.1 due to normalization issues with jsonschema <= 4.9 (where the optional dependency was named format_nongpl).

[kloczek]

As it is listed on top I'm using jsonschema 4.17.3. Yes .. reported result is without installed rfc3339-validator and rfc3986-validator. Should I add those modules to my build env? 🤔

[kevin-bates]

As it is listed on top I'm using jsonschema 4.17.3.

Sorry, I missed that.

I don't know how these "build env" are seeded, but jupyter_events depends on jsonschema[format-nongpl], so the equivalent of pip install "jsonschema[format-nongpl]" is required (and is the mechanism by which those validators get installed).

[kloczek]

Looks like 0.9.0 is OK now

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-events-0.9.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-events-0.9.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network'
============================= test session starts ==============================
platform linux -- Python 3.8.18, pytest-7.4.3, pluggy-1.3.0
rootdir: /home/tkloczko/rpmbuild/BUILD/jupyter_events-0.9.0
configfile: pyproject.toml
testpaths: tests/
plugins: anyio-4.0.0, asyncio-0.21.1, console-scripts-1.4.1
asyncio: mode=auto
collected 48 items

tests/test_cli.py .......                                                [ 14%]
tests/test_listeners.py .....                                            [ 25%]
tests/test_logger.py ...............                                     [ 56%]
tests/test_modifiers.py ...                                              [ 62%]
tests/test_schema.py ...............                                     [ 93%]
tests/test_traits.py ...                                                 [100%]

============================= slowest 10 durations =============================
0.31s call     tests/test_cli.py::test_cli_version[subprocess]
0.30s call     tests/test_cli.py::test_cli_good_raw[subprocess]
0.30s call     tests/test_cli.py::test_cli_invalid[subprocess]
0.30s call     tests/test_cli.py::test_cli_good[subprocess]
0.29s call     tests/test_cli.py::test_cli_malformed[subprocess]
0.29s call     tests/test_cli.py::test_cli_missing[subprocess]
0.29s call     tests/test_cli.py::test_cli_help[subprocess]
0.01s setup    tests/test_listeners.py::test_listener_function
0.01s setup    tests/test_listeners.py::test_listener_function_str_annotations
0.01s setup    tests/test_listeners.py::test_remove_listener_function
============================== 48 passed in 2.44s ==============================

Thank you 👍 Closing.