2.4.3: pytest is failing in some units

[kloczek]

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix> using 'installer` module
• run pytest with $PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>
• build is performed in env which is cut off from access to the public network (pytest is executed with -m "not network")

List of installed modules in build env:

```console Package Version ----------------------------- --------- alabaster 0.7.13 anyio 4.2.0 Babel 2.14.0 build 1.0.3 certifi 2023.7.22 charset-normalizer 3.3.2 cppclean 0.13 distro 1.8.0 dnf 4.18.2 docutils 0.20.1 exceptiongroup 1.1.3 gpg 1.23.2 h11 0.14.0 httpcore 1.0.0 httpx 0.25.2 idna 3.6 imagesize 1.4.1 importlib-metadata 7.0.1 iniconfig 2.0.0 installer 0.7.0 Jinja2 3.1.2 libdnf 0.72.0 markdown-it-py 3.0.0 MarkupSafe 2.1.3 mdit-py-plugins 0.4.0 mdurl 0.1.2 myst-parser 2.0.0 packaging 23.2 pluggy 1.3.0 Pygments 2.17.2 pyproject_hooks 1.0.0 pytest 7.4.3 pytest-asyncio 0.23.2 pytest-mock 3.12.0 python-dateutil 2.8.2 pytz 2023.3 PyYAML 6.0.1 requests 2.31.0 setuptools 69.0.3 setuptools-scm 8.0.4 setuptools-scm-git-archive 1.4 six 1.16.0 sniffio 1.3.0 snowballstemmer 2.2.0 Sphinx 7.1.2 sphinxcontrib-applehelp 1.0.4 sphinxcontrib-devhelp 1.0.5 sphinxcontrib-htmlhelp 2.0.4 sphinxcontrib-jsmath 1.0.1 sphinxcontrib-qthelp 1.0.3 sphinxcontrib-serializinghtml 1.1.9 tomli 2.0.1 typing_extensions 4.9.0 urllib3 1.26.18 wheel 0.42.0 zipp 3.17.0 ```

Please let me know if you need more details or want me to perform some diagnostics.

[kloczek]

Here is summary pytest output:

```console =========================== short test summary info ============================ ERROR tests/http/test_protocol_handler.py::TestHttpProtocolHandlerWithoutServerMock::test_proxy_connection_failed ERROR tests/http/test_protocol_handler.py::TestHttpProtocolHandlerWithoutServerMock::test_proxy_connection_failed ERROR tests/http/test_protocol_handler.py::TestHttpProtocolHandlerWithoutServerMock::test_proxy_authentication_failed ERROR tests/http/test_protocol_handler.py::TestHttpProtocolHandlerWithoutServerMock::test_proxy_authentication_failed ERROR tests/http/test_protocol_handler.py::TestHttpProtocolHandlerWithoutServerMock::test_proxy_bails_out_for_unknown_schemes ERROR tests/http/test_protocol_handler.py::TestHttpProtocolHandlerWithoutServerMock::test_proxy_bails_out_for_unknown_schemes ERROR tests/http/test_protocol_handler.py::TestHttpProtocolHandlerWithoutServerMock::test_proxy_bails_out_for_sip_request_lines ERROR tests/http/test_protocol_handler.py::TestHttpProtocolHandlerWithoutServerMock::test_proxy_bails_out_for_sip_request_lines ERROR tests/http/test_protocol_handler.py::TestHttpProtocolHandler::test_http_get ERROR tests/http/test_protocol_handler.py::TestHttpProtocolHandler::test_http_get ERROR tests/http/test_protocol_handler.py::TestHttpProtocolHandler::test_http_tunnel ERROR tests/http/test_protocol_handler.py::TestHttpProtocolHandler::test_http_tunnel ERROR tests/http/test_protocol_handler.py::TestHttpProtocolHandler::test_authenticated_proxy_http_get ERROR tests/http/test_protocol_handler.py::TestHttpProtocolHandler::test_authenticated_proxy_http_get ERROR tests/http/test_protocol_handler.py::TestHttpProtocolHandler::test_authenticated_proxy_http_tunnel ERROR tests/http/test_protocol_handler.py::TestHttpProtocolHandler::test_authenticated_proxy_http_tunnel ERROR tests/http/exceptions/test_http_proxy_auth_failed.py::TestHttpProxyAuthFailed::test_proxy_auth_fails_without_cred ERROR tests/http/exceptions/test_http_proxy_auth_failed.py::TestHttpProxyAuthFailed::test_proxy_auth_fails_without_cred ERROR tests/http/exceptions/test_http_proxy_auth_failed.py::TestHttpProxyAuthFailed::test_proxy_auth_fails_with_invalid_cred ERROR tests/http/exceptions/test_http_proxy_auth_failed.py::TestHttpProxyAuthFailed::test_proxy_auth_fails_with_invalid_cred ERROR tests/http/exceptions/test_http_proxy_auth_failed.py::TestHttpProxyAuthFailed::test_proxy_auth_works_with_valid_cred ERROR tests/http/exceptions/test_http_proxy_auth_failed.py::TestHttpProxyAuthFailed::test_proxy_auth_works_with_valid_cred ERROR tests/http/exceptions/test_http_proxy_auth_failed.py::TestHttpProxyAuthFailed::test_proxy_auth_works_with_mixed_case_basic_string ERROR tests/http/exceptions/test_http_proxy_auth_failed.py::TestHttpProxyAuthFailed::test_proxy_auth_works_with_mixed_case_basic_string ERROR tests/http/proxy/test_http_proxy.py::TestHttpProxyPlugin::test_proxy_plugin_on_and_before_upstream_connection ERROR tests/http/proxy/test_http_proxy.py::TestHttpProxyPlugin::test_proxy_plugin_on_and_before_upstream_connection ERROR tests/http/proxy/test_http_proxy.py::TestHttpProxyPlugin::test_proxy_plugin_before_upstream_connection_can_teardown ERROR tests/http/proxy/test_http_proxy.py::TestHttpProxyPlugin::test_proxy_plugin_before_upstream_connection_can_teardown ERROR tests/http/proxy/test_http_proxy_tls_interception.py::TestHttpProxyTlsInterception::test_e2e ERROR tests/http/web/test_web_server.py::TestWebServerPluginWithPacFilePlugin::test_pac_file_served_from_disk[/home/tkloczko/rpmbuild/BUILD/proxy.py-2.4.3/helper/proxy.pac] ERROR tests/http/web/test_web_server.py::TestWebServerPluginWithPacFilePlugin::test_pac_file_served_from_disk[/home/tkloczko/rpmbuild/BUILD/proxy.py-2.4.3/helper/proxy.pac] ERROR tests/http/web/test_web_server.py::TestWebServerPluginWithPacFilePlugin::test_pac_file_served_from_disk[function FindProxyForURL(url, host) { return "PROXY localhost:8899; DIRECT"; }] ERROR tests/http/web/test_web_server.py::TestWebServerPluginWithPacFilePlugin::test_pac_file_served_from_disk[function FindProxyForURL(url, host) { return "PROXY localhost:8899; DIRECT"; }] ERROR tests/http/web/test_web_server.py::TestStaticWebServerPlugin::test_static_web_server_serves ERROR tests/http/web/test_web_server.py::TestStaticWebServerPlugin::test_static_web_server_serves ERROR tests/http/web/test_web_server.py::TestStaticWebServerPlugin::test_static_web_server_serves_404 ERROR tests/http/web/test_web_server.py::TestStaticWebServerPlugin::test_static_web_server_serves_404 ERROR tests/http/web/test_web_server.py::TestWebServerPlugin::test_default_web_server_returns_404 ERROR tests/http/web/test_web_server.py::TestWebServerPlugin::test_default_web_server_returns_404 ERROR tests/integration/test_integration.py::test_integration[--threaded] - s... ERROR tests/integration/test_integration.py::test_integration[--threadless --local-executor 0] ERROR tests/integration/test_integration.py::test_integration[--threadless] ERROR tests/integration/test_integration.py::test_https_integration[--threaded --key-file /tmp/certificates/https-key.pem --cert-file /tmp/certificates/https-signed-cert.pem] ERROR tests/integration/test_integration.py::test_https_integration[--threadless --local-executor 0 --key-file /tmp/certificates/https-key.pem --cert-file /tmp/certificates/https-signed-cert.pem] ERROR tests/integration/test_integration.py::test_https_integration[--threadless --key-file /tmp/certificates/https-key.pem --cert-file /tmp/certificates/https-signed-cert.pem] ERROR tests/integration/test_integration.py::test_integration_with_interception_flags[--threaded --ca-cert-file /tmp/certificates/ca-cert.pem --ca-key-file /tmp/certificates/ca-key.pem --ca-signing-key /tmp/certificates/ca-signing-key.pem] ERROR tests/integration/test_integration.py::test_integration_with_interception_flags[--threadless --local-executor 0 --ca-cert-file /tmp/certificates/ca-cert.pem --ca-key-file /tmp/certificates/ca-key.pem --ca-signing-key /tmp/certificates/ca-signing-key.pem] ERROR tests/integration/test_integration.py::test_integration_with_interception_flags[--threadless --ca-cert-file /tmp/certificates/ca-cert.pem --ca-key-file /tmp/certificates/ca-key.pem --ca-signing-key /tmp/certificates/ca-signing-key.pem] ERROR tests/integration/test_integration.py::test_modify_chunk_response_integration[--threaded --plugin proxy.plugin.ModifyChunkResponsePlugin --ca-cert-file /tmp/certificates/ca-cert-chunk.pem --ca-key-file /tmp/certificates/ca-key-chunk.pem --ca-signing-key /tmp/certificates/ca-signing-key-chunk.pem] ERROR tests/integration/test_integration.py::test_modify_chunk_response_integration[--threadless --local-executor 0 --plugin proxy.plugin.ModifyChunkResponsePlugin --ca-cert-file /tmp/certificates/ca-cert-chunk.pem --ca-key-file /tmp/certificates/ca-key-chunk.pem --ca-signing-key /tmp/certificates/ca-signing-key-chunk.pem] ERROR tests/integration/test_integration.py::test_modify_chunk_response_integration[--threadless --plugin proxy.plugin.ModifyChunkResponsePlugin --ca-cert-file /tmp/certificates/ca-cert-chunk.pem --ca-key-file /tmp/certificates/ca-key-chunk.pem --ca-signing-key /tmp/certificates/ca-signing-key-chunk.pem] ERROR tests/integration/test_integration.py::test_modify_post_response_integration[--threaded --plugin proxy.plugin.ModifyPostDataPlugin --ca-cert-file /tmp/certificates/ca-cert-post.pem --ca-key-file /tmp/certificates/ca-key-post.pem --ca-signing-key /tmp/certificates/ca-signing-key-post.pem] ERROR tests/integration/test_integration.py::test_modify_post_response_integration[--threadless --local-executor 0 --plugin proxy.plugin.ModifyPostDataPlugin --ca-cert-file /tmp/certificates/ca-cert-post.pem --ca-key-file /tmp/certificates/ca-key-post.pem --ca-signing-key /tmp/certificates/ca-signing-key-post.pem] ERROR tests/integration/test_integration.py::test_modify_post_response_integration[--threadless --plugin proxy.plugin.ModifyPostDataPlugin --ca-cert-file /tmp/certificates/ca-cert-post.pem --ca-key-file /tmp/certificates/ca-key-post.pem --ca-signing-key /tmp/certificates/ca-signing-key-post.pem] ERROR tests/plugin/test_http_proxy_plugins.py::TestHttpProxyPluginExamples::test_modify_post_data_plugin[test_modify_post_data_plugin] ERROR tests/plugin/test_http_proxy_plugins.py::TestHttpProxyPluginExamples::test_modify_post_data_plugin[test_modify_post_data_plugin] ERROR tests/plugin/test_http_proxy_plugins.py::TestHttpProxyPluginExamples::test_proposed_rest_api_plugin[test_proposed_rest_api_plugin] ERROR tests/plugin/test_http_proxy_plugins.py::TestHttpProxyPluginExamples::test_proposed_rest_api_plugin[test_proposed_rest_api_plugin] ERROR tests/plugin/test_http_proxy_plugins.py::TestHttpProxyPluginExamples::test_redirect_to_custom_server_plugin[test_redirect_to_custom_server_plugin] ERROR tests/plugin/test_http_proxy_plugins.py::TestHttpProxyPluginExamples::test_redirect_to_custom_server_plugin[test_redirect_to_custom_server_plugin] ERROR tests/plugin/test_http_proxy_plugins.py::TestHttpProxyPluginExamples::test_redirect_to_custom_server_plugin_skips_https[test_redirect_to_custom_server_plugin] ERROR tests/plugin/test_http_proxy_plugins.py::TestHttpProxyPluginExamples::test_redirect_to_custom_server_plugin_skips_https[test_redirect_to_custom_server_plugin] ERROR tests/plugin/test_http_proxy_plugins.py::TestHttpProxyPluginExamples::test_filter_by_upstream_host_plugin[test_filter_by_upstream_host_plugin] ERROR tests/plugin/test_http_proxy_plugins.py::TestHttpProxyPluginExamples::test_filter_by_upstream_host_plugin[test_filter_by_upstream_host_plugin] ERROR tests/plugin/test_http_proxy_plugins.py::TestHttpProxyPluginExamples::test_man_in_the_middle_plugin[test_man_in_the_middle_plugin] ERROR tests/plugin/test_http_proxy_plugins.py::TestHttpProxyPluginExamples::test_man_in_the_middle_plugin[test_man_in_the_middle_plugin] ERROR tests/plugin/test_http_proxy_plugins.py::TestHttpProxyPluginExamples::test_filter_by_url_regex_plugin[test_filter_by_url_regex_plugin] ERROR tests/plugin/test_http_proxy_plugins.py::TestHttpProxyPluginExamples::test_filter_by_url_regex_plugin[test_filter_by_url_regex_plugin] ERROR tests/plugin/test_http_proxy_plugins.py::TestHttpProxyPluginExamples::test_shortlink_plugin[test_shortlink_plugin] ERROR tests/plugin/test_http_proxy_plugins.py::TestHttpProxyPluginExamples::test_shortlink_plugin[test_shortlink_plugin] ERROR tests/plugin/test_http_proxy_plugins.py::TestHttpProxyPluginExamples::test_shortlink_plugin_unknown[test_shortlink_plugin] ERROR tests/plugin/test_http_proxy_plugins.py::TestHttpProxyPluginExamples::test_shortlink_plugin_unknown[test_shortlink_plugin] ERROR tests/plugin/test_http_proxy_plugins.py::TestHttpProxyPluginExamples::test_shortlink_plugin_external[test_shortlink_plugin] ERROR tests/plugin/test_http_proxy_plugins.py::TestHttpProxyPluginExamples::test_shortlink_plugin_external[test_shortlink_plugin] ERROR tests/plugin/test_http_proxy_plugins_with_tls_interception.py::TestHttpProxyPluginExamplesWithTlsInterception::test_modify_post_data_plugin[test_modify_post_data_plugin] ERROR tests/plugin/test_http_proxy_plugins_with_tls_interception.py::TestHttpProxyPluginExamplesWithTlsInterception::test_modify_post_data_plugin[test_modify_post_data_plugin] ERROR tests/plugin/test_http_proxy_plugins_with_tls_interception.py::TestHttpProxyPluginExamplesWithTlsInterception::test_man_in_the_middle_plugin[test_man_in_the_middle_plugin] ERROR tests/plugin/test_http_proxy_plugins_with_tls_interception.py::TestHttpProxyPluginExamplesWithTlsInterception::test_man_in_the_middle_plugin[test_man_in_the_middle_plugin] ======================= 186 passed, 78 errors in 13.13s ======================== Exception ignored in: Traceback (most recent call last): File "/usr/lib64/python3.8/asyncio/base_events.py", line 656, in __del__ self.close() File "/usr/lib64/python3.8/asyncio/unix_events.py", line 58, in close super().close() File "/usr/lib64/python3.8/asyncio/selector_events.py", line 92, in close self._close_self_pipe() File "/usr/lib64/python3.8/asyncio/selector_events.py", line 99, in _close_self_pipe self._remove_reader(self._ssock.fileno()) File "/usr/lib64/python3.8/asyncio/selector_events.py", line 280, in _remove_reader mask, (reader, writer) = key.events, key.data ValueError: not enough values to unpack (expected 2, got 0) Exception ignored in: Traceback (most recent call last): File "/usr/lib64/python3.8/asyncio/base_events.py", line 656, in __del__ self.close() File "/usr/lib64/python3.8/asyncio/unix_events.py", line 58, in close super().close() File "/usr/lib64/python3.8/asyncio/selector_events.py", line 92, in close self._close_self_pipe() File "/usr/lib64/python3.8/asyncio/selector_events.py", line 99, in _close_self_pipe self._remove_reader(self._ssock.fileno()) File "/usr/lib64/python3.8/asyncio/selector_events.py", line 280, in _remove_reader mask, (reader, writer) = key.events, key.data ValueError: not enough values to unpack (expected 2, got 0) Exception ignored in: Traceback (most recent call last): File "/usr/lib64/python3.8/asyncio/base_events.py", line 656, in __del__ self.close() File "/usr/lib64/python3.8/asyncio/unix_events.py", line 58, in close super().close() File "/usr/lib64/python3.8/asyncio/selector_events.py", line 92, in close self._close_self_pipe() File "/usr/lib64/python3.8/asyncio/selector_events.py", line 99, in _close_self_pipe self._remove_reader(self._ssock.fileno()) File "/usr/lib64/python3.8/asyncio/selector_events.py", line 280, in _remove_reader mask, (reader, writer) = key.events, key.data ValueError: not enough values to unpack (expected 2, got 0) Exception ignored in: Traceback (most recent call last): File "/usr/lib64/python3.8/asyncio/base_events.py", line 656, in __del__ self.close() File "/usr/lib64/python3.8/asyncio/unix_events.py", line 58, in close super().close() File "/usr/lib64/python3.8/asyncio/selector_events.py", line 92, in close self._close_self_pipe() File "/usr/lib64/python3.8/asyncio/selector_events.py", line 99, in _close_self_pipe self._remove_reader(self._ssock.fileno()) File "/usr/lib64/python3.8/asyncio/selector_events.py", line 280, in _remove_reader mask, (reader, writer) = key.events, key.data ValueError: not enough values to unpack (expected 2, got 0) ```

And here it is full pytest output: python-proxy-py.FAIL.txt

Do I have something missing in build env? 🤔

[kloczek]

BTW looks like in some units names are used spaces and current directory name. All that makes difficult to add failing units names to --deselect list.

[kloczek]

Gentle ping .. any update? 🤔

[abhinavsingh]

@kloczek Can you try the develop branch once again. Over past few days I updated repo dependencies, due to which workflows were also failing on GitHub. Want to see if those changes helps your case.

[kloczek]

One sec .. it will take me ~1-2h to perform full checks as I have build system queue a bit clogged. Will back with results ASP 😋

[kloczek]

Just tested 2.4.4 and issue still is not resolved Full pytest output is in attachment python-proxy-py.FAIL.txt

[abhinavsingh]

Just tested 2.4.4 and issue still is not resolved Full pytest output is in attachment python-proxy-py.FAIL.txt

Thank you for the logs. From the looks of it, I get an intuition that python3.10 on your system is somehow defaulting to _UnixSelectorEventLoop. This in general can be fixed by adding to the code:

asyncio.set_event_loop_policy(asyncio.DefaultEventLoopPolicy())

Can you try to add this line and see if it helps?

Thank you

[kloczek]

Where I need to add that? 🤔

[abhinavsingh]

Where I need to add that? 🤔

I would suggest try adding within proxy/__init__.py or within setUp of the test that fails for you. Lemme know if this does the trick and I'll be happy to include this within the repo.

[kloczek]

Please drop the patch which you want me to test.