Low level crash on Windows after introducing mocket

[LilyFoote]

I'm seeing a Windows fatal exception: access violation in CI after replacing httpretty with mocket. Full traceback:

Windows fatal exception: access violation

Current thread 0x000017c0 (most recent call first):
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\magic\compat.py", line 189 in load
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\magic\compat.py", line 242 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 940 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1147 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1176 in _find_and_load
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1232 in _handle_fromlist
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\magic\__init__.py", line 433 in _add_compat
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\magic\__init__.py", line 469 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 940 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1147 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1176 in _find_and_load
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\mocket\mockhttp.py", line 12 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 940 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1147 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1176 in _find_and_load
  File "D:\a\kolo\kolo\python\tests\filters\test_requests.py", line 14 in <module>
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\_pytest\assertion\rewrite.py", line 178 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1147 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1176 in _find_and_load
  File "<frozen importlib._bootstrap>", line 1204 in _gcd_import
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\importlib\__init__.py", line 126 in import_module
Current thread 0x000017c0 (most recent call first):
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\magic\compat.py", line 189 in load
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\magic\compat.py", line 242 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 940 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1147 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1176 in _find_and_load
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1232 in _handle_fromlist
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\magic\__init__.py", line 433 in _add_compat
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\magic\__init__.py", line 469 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 940 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1147 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1176 in _find_and_load
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\mocket\mockhttp.py", line 12 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 940 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1147 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1176 in _find_and_load
  File "D:\a\kolo\kolo\python\tests\filters\test_requests.py", line 14 in <module>
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\_pytest\assertion\rewrite.py", line 178 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1147 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1176 in _find_and_load
  File "<frozen importlib._bootstrap>", line 1204 in _gcd_import
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\importlib\__init__.py", line 126 in import_module
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\_pytest\pathlib.py", line 591 in import_path
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\_pytest\python.py", line 492 in importtestmodule
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\_pytest\python.py", line 545 in _getobj
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\_pytest\python.py", line 287 in obj
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\pytest_asyncio\plugin.py", line 640 in _patched_collect
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\_pytest\runner.py", line 389 in collect
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\_pytest\runner.py", line 341 in from_call
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\_pytest\runner.py", line 391 in pytest_make_collect_report
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\pluggy\_callers.py", line 103 in _multicall
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\pluggy\_manager.py", line 120 in _hookexec
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\pluggy\_hooks.py", line 513 in __call__
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\_pytest\runner.py", line 567 in collect_one_node
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\_pytest\main.py", line 837 in _collect_one_node
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\_pytest\main.py", line 974 in genitems
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\_pytest\main.py", line 979 in genitems
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\_pytest\main.py", line 979 in genitems
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\_pytest\main.py", line 979 in genitems
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\_pytest\main.py", line 811 in perform_collect
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\_pytest\main.py", line 349 in pytest_collection
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\pluggy\_callers.py", line 103 in _multicall
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\pluggy\_manager.py", line 120 in _hookexec
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\pluggy\_hooks.py", line 513 in __call__
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\_pytest\main.py", line 338 in _main
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\_pytest\main.py", line 285 in wrap_session
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\_pytest\main.py", line 332 in pytest_cmdline_main
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\pluggy\_callers.py", line 103 in _multicall
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\pluggy\_manager.py", line 120 in _hookexec
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\pluggy\_hooks.py", line 513 in __call__
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\_pytest\config\__init__.py", line 178 in main
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\_pytest\config\__init__.py", line 206 in console_main
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\pytest\__main__.py", line 7 in <module>
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\coverage\execfile.py", line 211 in run
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\coverage\cmdline.py", line 858 in do_run
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\coverage\cmdline.py", line 681 in command_line
  File "D:\a\kolo\kolo\python\.venv\Lib\site-packages\coverage\cmdline.py", line 970 in main
  File "D:\a\kolo\kolo\python\.venv\Scripts\coverage.exe\__main__.py", line 7 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main

[mindflayer]

It looks like something related to python-magic and I read from its README you have to:

# You'll need DLLs for libmagic. @julian-r maintains a pypi package with the DLLs, you can fetch it with:

pip install python-magic-bin

This said, AFAIK you are the first Windows user and I am not 100% sure it'll work anyway because of another binary dependency: httptools.

[mindflayer]

I am kinda thinking out-loud here: the need for the first dependency is definitely something we could somehow overcome. It'll be a bit more complex for the second one, but probably not impossible. I believe we'll be able to find HTTP parsers implemented in pure Python, even as part of the standard library. The thing is: I'd definitely need someone else help for adding Windows support. The latest Windows version I used was XP.

[LilyFoote]

Thanks! Installing that dependency has done the trick!

There doesn't appear to be any other changes needed for my tests to pass on Windows.

[mindflayer]

Does it mean that was the only necessary step to make it work with Windows? Should I close this issue?

[LilyFoote]

We could add the extra package as a conditional dependency for Windows users, though I think that should really live upstream.

[mindflayer]

Yeah, it should be responsibility of the module. It should install what's needed to work on every platform, definitely not Mocket's responsibility. I'm open to merge a PR with the change, anyway.

[LilyFoote]

I opened the PR upstream: https://github.com/ahupp/python-magic/pull/332