tarpas
commented
1 year ago testmon includes pytest rootdir in it's coverage and excludes sys.prefix . Including everything is prohibitive performance wise and I don't have a clear idea how to support multiple directories.
Also we're working on a feature of sharing testmondata across systems and I don't have a clear idea how to support python paths which are portable across systems.
coverage deals with this problems in it's way but the merging of files is clumsy and slow, so not usable for acceleration tool like testmon.
For the coming months the answer, I'm afraid, is that testmon only supports editable installs where everything you need to track is under pytest rootdir.
adkinsjd
We (@jacksongoode and I) are trying to use testmon in a managed environment (Databricks jobs) that requires all code be contained either within a single python script or be installed as a package. To do so we package the code we want to test, install it, then upload and run a script that invokes pytest with testmon and sets the pytest rootdir to the site-packages directory of package we are testing.
In this setup the correct tests are collected and run, but testmon doesn't detect changes in the package code, only the tests themselves. Is there a way to configure pytest/testmon/the environment such that testmon can detect changes to the code of a specific installed package.
More broadly, how does testmon choose which directories to analyze with coverage? Is there some way, either in current testmon, or as a new feature, to add a directory to what testmon considers the "source" code?
We've tried a number of pytest configurations including changing the rootdir and testpaths for pytest. We have considered, but have not tried manually manipulating the environment in our testing script, perhaps modifying the python path. It's hard to know what to do without a deeper understanding of how testmon selects which directories to check for changes and how testmon excludes installed packages from those checks.