Integration test example in bootstrapped project (dbx init) fails due to pytest dependency

[alanmazankiewicz]

Expected Behavior

Integration test should run successfully

Current Behavior

Integration test fails with:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<command--1> in <cell line: 3>()
      1 import sys
      2
----> 3 import pytest
      4
      5 if __name__ == "__main__":

/databricks/python_shell/dbruntime/PythonPackageImportsInstrumentation/__init__.py in import_patch(name, globals,
locals, fromlist, level)
    169             # Import the desired module. If you’re seeing this while debugging a failed import,
    170             # look at preceding stack frames for relevant error information.
--> 171             original_result = python_builtin_import(name, globals, locals, fromlist, level)
    172
    173             is_root_import = thread_local._nest_level == 1

ModuleNotFoundError: No module named 'pytest'

In setup.py pytest is part of the test requirements which get not added as a dependency on dbx deploy / dbx execute. Pytest is also not part of DBR as suggested by the comment in setup.py

Steps to Reproduce (for bugs)

dbx init my-proj dbx execute my-proj-sample-tests --cluster-name="<CLUSTER_NAME>"

Related

• https://github.com/databrickslabs/dbx/issues/514

dbx version used

0.8.9

[RaccoonForever]

Hey guys, have the same problem. If we install it directly on the cluster we have another error due to arguments wrong that are applied to python and not pytest. I don't understand how this should work since python doesn't accept "cov" argument for example.

EDIT: the error

EDIT: I made it work:

• in setup.py replace PACKAGE_REQUIREMENTS by PACKAGE_REQUIREMENTS = ["pyyaml", "pytest", "pytest-cov"]

Of course we don't want pytest and pytest-cov to be installed with the package in a production environment. But for now it will do.

Even if "--requirement-file" option is deprecated I would use that with a specific requirements for each usage right now if I wanted it perfect.