Currently, the presence of mock_anki_modules() in test/config/config_settings_spec.py causes the anki imports to be mocked, for all tests (including those that don't call mock_anki_modules(), when the tests are run together.
To reproduce
Steps
Install all dependencies (including dev):
pipenv install --dev
Run all mamba tests together:
pipenv run mamba ./
Run the tests in test/export/note_sorter_spec.py by themselves:
pipenv run mamba ./test/export/note_sorter_spec.py
Expected result
Both for 2 and 3 all tests pass.
Actual result
For 2 all tests pass:
31 examples ran (3 pending) in 1.2636 seconds
For 3 the tests crash:
```
Traceback (most recent call last):
File "/home/x/.local/share/virtualenvs/CrowdAnki-4rfLFX4y/bin/mamba", line 8, in
sys.exit(main())
File "/home/x/.local/share/virtualenvs/CrowdAnki-4rfLFX4y/lib/python3.7/site-packages/mamba/cli.py", line 18, in main
runner.run()
File "/home/x/.local/share/virtualenvs/CrowdAnki-4rfLFX4y/lib/python3.7/site-packages/mamba/runners.py", line 29, in run
modules = self.example_collector.modules()
File "/home/x/.local/share/virtualenvs/CrowdAnki-4rfLFX4y/lib/python3.7/site-packages/mamba/example_collector.py", line 20, in modules
with self._load_module_from(path) as module:
File "/usr/local/lib/python3.7/contextlib.py", line 112, in __enter__
return next(self.gen)
File "/home/x/.local/share/virtualenvs/CrowdAnki-4rfLFX4y/lib/python3.7/site-packages/mamba/example_collector.py", line 52, in _load_module_from
yield self._module_from_ast(name, path)
File "/home/x/.local/share/virtualenvs/CrowdAnki-4rfLFX4y/lib/python3.7/site-packages/mamba/example_collector.py", line 70, in _module_from_ast
exec(code, module.__dict__)
File "./test/export/note_sorter_spec.py", line 5, in
from aqt import mw
File "/home/x/.local/share/virtualenvs/CrowdAnki-4rfLFX4y/lib/python3.7/site-packages/aqt/__init__.py", line 22, in
from aqt.qt import *
File "/home/x/.local/share/virtualenvs/CrowdAnki-4rfLFX4y/lib/python3.7/site-packages/aqt/qt.py", line 16, in
from PyQt5.QtWebEngineWidgets import * # type: ignore
ModuleNotFoundError: No module named 'PyQt5.QtWebEngineWidgets'
```
(The fact that the tests crash is another issue, but they should either always crash or always succeed, so the fact that their completion depends on the way they're called is problematic.)
Alternative approach (to make sure)
To reproduce
In test/config/config_settings_spec.py (the file containing the mock_anki_modules() call), comment out everything after mock_anki_modules(). (We're not interested in the behaviour of config_settings_spec.py in the absence of module mocking.)
Run all mamba tests:
pipenv run mamba ./
Comment out mock_anki_modules() in test/config/config_settings_spec.py.
Run all mamba tests again.
Expected result
The same result is returned in 2 and 4.
Actual result
For 2 the tests all succeed:
25 examples ran (3 pending) in 1.2638 seconds
For 4 the tests crashed again.
Presumably, given that config comes before export alphabetically, the patching of sys.module persists throughout the testing.
(I think that I haven't missed anything obvious, for why the behaviour is different.)
The main short-term conclusion (mainly for me) is that the behaviour of the tests is different, depending on whether they're called individually, or all together.
(This is mostly to remind myself for the future.)
Currently, the presence of
mock_anki_modules()intest/config/config_settings_spec.pycauses the anki imports to be mocked, for all tests (including those that don't callmock_anki_modules(), when the tests are run together.To reproduce
Steps
test/export/note_sorter_spec.pyby themselves:Expected result
Both for 2 and 3 all tests pass.
Actual result
For 2 all tests pass:
For 3 the tests crash:
(The fact that the tests crash is another issue, but they should either always crash or always succeed, so the fact that their completion depends on the way they're called is problematic.)
Alternative approach (to make sure)
To reproduce
In
test/config/config_settings_spec.py(the file containing themock_anki_modules()call), comment out everything aftermock_anki_modules(). (We're not interested in the behaviour ofconfig_settings_spec.pyin the absence of module mocking.)Run all mamba tests:
Comment out
mock_anki_modules()intest/config/config_settings_spec.py.Run all mamba tests again.
Expected result
The same result is returned in 2 and 4.
Actual result
For 2 the tests all succeed:
For 4 the tests crashed again.
Presumably, given that
configcomes beforeexportalphabetically, the patching ofsys.modulepersists throughout the testing.(I think that I haven't missed anything obvious, for why the behaviour is different.)
The main short-term conclusion (mainly for me) is that the behaviour of the tests is different, depending on whether they're called individually, or all together.