1.6.0: pytest is failing

[kloczek]

I'm trying to package your module as rpm packag. So I'm using typical in such case build, install and test cycle used on building package from non-root account:

• "setup.py build"
• "setup.py install --root </install/prefix>"
• "pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

May I ask for help because few units are failing:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-cloudpickle-1.6.0-6.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-cloudpickle-1.6.0-6.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra --import-mode=importlib
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.11, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
Using --randomly-seed=739258526
rootdir: /home/tkloczko/rpmbuild/BUILD/cloudpickle-1.6.0, configfile: tox.ini
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, freezegun-0.4.2, aspectlib-1.5.2, toolbox-0.5, rerunfailures-9.1.1, requests-mock-1.9.3, cov-2.12.1, pyfakefs-4.5.0, flaky-3.7.0, benchmark-3.4.1, xdist-2.3.0, pylama-7.7.1, datadir-1.3.1, regressions-2.2.0, cases-3.6.3, xprocess-0.18.1, black-0.3.12, anyio-3.3.0, asyncio-0.15.1, trio-0.7.0, httpbin-1.0.0, subtests-0.5.0, isort-2.0.0, hypothesis-6.14.6, mock-3.6.1, profiling-1.7.0, randomly-3.8.0, Faker-8.12.1
collected 7 items / 2 errors / 5 selected

================================================================================== ERRORS ==================================================================================
________________________________________________________________ ERROR collecting tests/cloudpickle_test.py ________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/cloudpickle-1.6.0/tests/cloudpickle_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/cloudpickle_test.py:52: in <module>
    from .testutils import subprocess_pickle_echo
E   ImportError: attempted relative import with no known parent package
______________________________________________________________ ERROR collecting tests/test_backward_compat.py ______________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/cloudpickle-1.6.0/tests/test_backward_compat.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/test_backward_compat.py:17: in <module>
    from .generate_old_pickles import PICKLE_DIRECTORY
E   ImportError: attempted relative import with no known parent package
========================================================================= short test summary info ==========================================================================
ERROR tests/cloudpickle_test.py
ERROR tests/test_backward_compat.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================ 2 errors in 0.49s =============================================================================
pytest-xprocess reminder::Be sure to terminate the started process by running 'pytest --xkill' if you have not explicitly done so in your fixture with 'xprocess.getinfo(<process_name>).terminate()'.

[kloczek]

After add to ignore list tests/cloudpickle_test.py rest seems is OK.

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-cloudpickle-1.6.0-6.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-cloudpickle-1.6.0-6.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra --ignore tests/cloudpickle_test.py
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.11, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
Using --randomly-seed=646853930
rootdir: /home/tkloczko/rpmbuild/BUILD/cloudpickle-1.6.0, configfile: tox.ini
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, freezegun-0.4.2, aspectlib-1.5.2, toolbox-0.5, rerunfailures-9.1.1, requests-mock-1.9.3, cov-2.12.1, pyfakefs-4.5.0, flaky-3.7.0, benchmark-3.4.1, xdist-2.3.0, pylama-7.7.1, datadir-1.3.1, regressions-2.2.0, cases-3.6.3, xprocess-0.18.1, black-0.3.12, anyio-3.3.0, asyncio-0.15.1, trio-0.7.0, httpbin-1.0.0, subtests-0.5.0, isort-2.0.0, hypothesis-6.14.6, mock-3.6.1, profiling-1.7.0, randomly-3.8.0, Faker-8.12.1
collected 14 items

tests/test_backward_compat.py .......
tests/cloudpickle_file_test.py .......

============================================================================ 14 passed in 0.29s ============================================================================
pytest-xprocess reminder::Be sure to terminate the started process by running 'pytest --xkill' if you have not explicitly done so in your fixture with 'xprocess.getinfo(<process_name>).terminate()'.

[jrbourbeau]

Just a shot in the dark, but you might try adding the tests directory to PYTHONPATH as documented here https://github.com/cloudpipe/cloudpickle#running-the-tests

[kloczek]

Just had a look one more time on that and here is the patch which allows use pytest without this kind of DIY procedures

--- a/tests/cloudpickle_testpkg/_cloudpickle_testpkg/mod.py~    2020-08-25 23:21:22.000000000 +0100
+++ b/tests/cloudpickle_testpkg/_cloudpickle_testpkg/mod.py     2021-09-12 00:43:12.110364370 +0100
@@ -20,19 +20,19 @@
 # module.  The following lines emulate such a behavior without being a compiled
 # extension module.

-submodule_name = '_cloudpickle_testpkg.mod.dynamic_submodule'
+submodule_name = 'tests.cloudpickle_testpkg._cloudpickle_testpkg.mod.dynamic_submodule'
 dynamic_submodule = types.ModuleType(submodule_name)

 # This line allows the dynamic_module to be imported using either one of:
-# - ``from _cloudpickle_testpkg.mod import dynamic_submodule``
-# - ``import _cloudpickle_testpkg.mod.dynamic_submodule``
+# - ``from tests.cloudpickle_testpkg._cloudpickle_testpkg.mod import dynamic_submodule``
+# - ``import tests.cloudpickle_testpkg._cloudpickle_testpkg.mod.dynamic_submodule``
 sys.modules[submodule_name] = dynamic_submodule
 # Both lines will make importlib try to get the module from sys.modules after
 # importing the parent module, before trying getattr(mod, 'dynamic_submodule'),
 # so this dynamic module could be binded to another name. This behavior is
 # demonstrated with `dynamic_submodule_two`

-submodule_name_two = '_cloudpickle_testpkg.mod.dynamic_submodule_two'
+submodule_name_two = 'tests.cloudpickle_testpkg._cloudpickle_testpkg.mod.dynamic_submodule_two'
 # Notice the inconsistent name binding, breaking attribute lookup-based import
 # attempts.
 another_submodule = types.ModuleType(submodule_name_two)
@@ -42,7 +42,7 @@
 # In this third case, the module is not added to sys.modules, and can only be
 # imported using attribute lookup-based imports.
 submodule_three = types.ModuleType(
-    '_cloudpickle_testpkg.mod.dynamic_submodule_three'
+    'tests.cloudpickle_testpkg._cloudpickle_testpkg.mod.dynamic_submodule_three'
 )
 code = """
 def f(x):
@@ -54,7 +54,7 @@
 # What about a dynamic submodule inside a dynamic submodule inside an
 # importable module?
 subsubmodule_name = (
-    '_cloudpickle_testpkg.mod.dynamic_submodule.dynamic_subsubmodule'
+    'tests.cloudpickle_testpkg._cloudpickle_testpkg.mod.dynamic_submodule.dynamic_subsubmodule'
 )
 dynamic_subsubmodule = types.ModuleType(subsubmodule_name)
 dynamic_submodule.dynamic_subsubmodule = dynamic_subsubmodule
--- a/tests/cloudpickle_test.py~        2020-08-25 23:21:22.000000000 +0100
+++ b/tests/cloudpickle_test.py 2021-09-12 00:40:30.385885184 +0100
@@ -53,7 +53,7 @@
 from .testutils import assert_run_python_script
 from .testutils import subprocess_worker

-from _cloudpickle_testpkg import relative_imports_factory
+from tests.cloudpickle_testpkg._cloudpickle_testpkg import relative_imports_factory

 _TEST_GLOBAL_VARIABLE = "default_value"
@@ -721,22 +721,22 @@
         # their parent modules are considered importable by cloudpickle.
         # See the mod_with_dynamic_submodule documentation for more
         # details of this use case.
-        import _cloudpickle_testpkg.mod.dynamic_submodule as m
+        import tests.cloudpickle_testpkg._cloudpickle_testpkg.mod.dynamic_submodule as m
         assert _is_importable(m)
         assert pickle_depickle(m, protocol=self.protocol) is m

         # Check for similar behavior for a module that cannot be imported by
         # attribute lookup.
-        from _cloudpickle_testpkg.mod import dynamic_submodule_two as m2
-        # Note: import _cloudpickle_testpkg.mod.dynamic_submodule_two as m2
+        from tests.cloudpickle_testpkg._cloudpickle_testpkg.mod import dynamic_submodule_two as m2
+        # Note: import tests.cloudpickle_testpkg._cloudpickle_testpkg.mod.dynamic_submodule_two as m2
         # works only for Python 3.7+
         assert _is_importable(m2)
         assert pickle_depickle(m2, protocol=self.protocol) is m2

         # Submodule_three is a dynamic module only importable via module lookup
         with pytest.raises(ImportError):
-            import _cloudpickle_testpkg.mod.submodule_three  # noqa
-        from _cloudpickle_testpkg.mod import submodule_three as m3
+            import tests.cloudpickle_testpkg._cloudpickle_testpkg.mod.submodule_three  # noqa
+        from tests.cloudpickle_testpkg._cloudpickle_testpkg.mod import submodule_three as m3
         assert not _is_importable(m3)

         # This module cannot be pickled using attribute lookup (as it does not
@@ -748,7 +748,7 @@

         # Do the same for an importable dynamic submodule inside a dynamic
         # module inside a file-backed module.
-        import _cloudpickle_testpkg.mod.dynamic_submodule.dynamic_subsubmodule as sm  # noqa
+        import tests.cloudpickle_testpkg._cloudpickle_testpkg.mod.dynamic_submodule.dynamic_subsubmodule as sm  # noqa
         assert _is_importable(sm)
         assert pickle_depickle(sm, protocol=self.protocol) is sm

@@ -2076,7 +2076,7 @@
     def test___reduce___returns_string(self):
         # Non regression test for objects with a __reduce__ method returning a
         # string, meaning "save by attribute using save_global"
-        from _cloudpickle_testpkg import some_singleton
+        from tests.cloudpickle_testpkg._cloudpickle_testpkg import some_singleton
         assert some_singleton.__reduce__() == "some_singleton"
         depickled_singleton = pickle_depickle(
             some_singleton, protocol=self.protocol)
@@ -2148,7 +2148,7 @@
         assert depickled_T1 is depickled_T2

     def test_pickle_importable_typevar(self):
-        from _cloudpickle_testpkg import T
+        from tests.cloudpickle_testpkg._cloudpickle_testpkg import T
         T1 = pickle_depickle(T, protocol=self.protocol)
         assert T1 is T

@@ -2296,12 +2296,12 @@

 def test_lookup_module_and_qualname_importable_typevar():
-    import _cloudpickle_testpkg
-    T = _cloudpickle_testpkg.T
+    import tests.cloudpickle_testpkg._cloudpickle_testpkg
+    T = tests.cloudpickle_testpkg._cloudpickle_testpkg.T
     module_and_name = _lookup_module_and_qualname(T, name=T.__name__)
     assert module_and_name is not None
     module, name = module_and_name
-    assert module is _cloudpickle_testpkg
+    assert module is tests.cloudpickle_testpkg._cloudpickle_testpkg
     assert name == 'T'

Please let me know if you want that patch as PR. Here is result:

+ /usr/bin/pytest -ra --pyargs tests
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.10.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
Using --randomly-seed=1569279798
rootdir: /home/tkloczko/rpmbuild/BUILD/cloudpickle-1.6.0, configfile: tox.ini
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, freezegun-0.4.2, aspectlib-1.5.2, toolbox-0.5, rerunfailures-9.1.1, requests-mock-1.9.3, cov-2.12.1, flaky-3.7.0, benchmark-3.4.1, xdist-2.3.0, pylama-7.7.1, datadir-1.3.1, regressions-2.2.0, cases-3.6.3, xprocess-0.18.1, black-0.3.12, asyncio-0.15.1, subtests-0.5.0, isort-2.0.0, hypothesis-6.14.6, mock-3.6.1, profiling-1.7.0, randomly-3.8.0, Faker-8.12.1, nose2pytest-1.0.8, pyfakefs-4.5.1, tornado-0.8.1, twisted-1.13.3, aiohttp-0.3.0, localserver-0.5.0, anyio-3.3.1
collected 231 items

tests/cloudpickle_test.py ..................................................................s......................................................................................................................................................
tests/test_backward_compat.py .......
tests/cloudpickle_file_test.py .......

========================================================================= short test summary info ==========================================================================
SKIPPED [1] tests/cloudpickle_test.py:2112: Need Pickle Protocol 5 or later
===================================================================== 230 passed, 1 skipped in 20.81s ======================================================================
pytest-xprocess reminder::Be sure to terminate the started process by running 'pytest --xkill' if you have not explicitly done so in your fixture with 'xprocess.getinfo(<process_name>).terminate()'.

[kloczek]

Sorry wrong result:

+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.10.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
Using --randomly-seed=297349340
rootdir: /home/tkloczko/rpmbuild/BUILD/cloudpickle-1.6.0, configfile: tox.ini
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, freezegun-0.4.2, aspectlib-1.5.2, toolbox-0.5, rerunfailures-9.1.1, requests-mock-1.9.3, cov-2.12.1, flaky-3.7.0, benchmark-3.4.1, xdist-2.3.0, pylama-7.7.1, datadir-1.3.1, regressions-2.2.0, cases-3.6.3, xprocess-0.18.1, black-0.3.12, asyncio-0.15.1, subtests-0.5.0, isort-2.0.0, hypothesis-6.14.6, mock-3.6.1, profiling-1.7.0, randomly-3.8.0, Faker-8.12.1, nose2pytest-1.0.8, pyfakefs-4.5.1, tornado-0.8.1, twisted-1.13.3, aiohttp-0.3.0, localserver-0.5.0, anyio-3.3.1
collected 231 items

tests/cloudpickle_file_test.py .......
tests/test_backward_compat.py .......
tests/cloudpickle_test.py .......................................................................................s.................................................................................................................................

========================================================================= short test summary info ==========================================================================
SKIPPED [1] tests/cloudpickle_test.py:2112: Need Pickle Protocol 5 or later
===================================================================== 230 passed, 1 skipped in 20.64s ======================================================================
pytest-xprocess reminder::Be sure to terminate the started process by running 'pytest --xkill' if you have not explicitly done so in your fixture with 'xprocess.getinfo(<process_name>).terminate()'.

[pierreglaser]

Hi @kloczek @jrbourbeau,

The issue reported here comes from the fact that cloudpickle uses a fully fledged test package (_cloudpickle_testpkg) to test some serialization patterns tied to modules/packages.

To run the test suite successfully, you have to install this said package using:

pip install /path/to/cloudpickle/tests/cloudpickle_testpkg

Admittedly, installing a test package is a bit manual, and I'm open to more automatic alternatives that preserves explicit pip installation.

Just a shot in the dark, but you might try adding the tests directory to PYTHONPATH as documented here https://github.com/cloudpipe/cloudpickle#running-the-tests

Although this instruction may solve the issue @kloczek is running against, this is purely coincidental, and not the setting that the currently failing cloudpickle tests are actually testing. This instruction was written 7 years ago, and is most likely outdated. This section could definitely use an update, which I (or anyone) can do, once dust has settled on a potential automation of the test package installation. Thanks for shedding light on this outdated section @jrbourbeau, and sorry to both of you for the confusion it may have caused!

[kloczek]

The issue reported here comes from the fact that cloudpickle uses a fully fledged test package (_cloudpickle_testpkg) to test some serialization patterns tied to modules/packages.

But if it is only part of internal test suite it does not make to much sense to install it before testing.

Fust updated my patch for 2.0.0.

# BUG: pytest is failing https://github.com/cloudpipe/cloudpickle/issues/436

--- a/tests/cloudpickle_testpkg/_cloudpickle_testpkg/mod.py~    2020-08-25 23:21:22.000000000 +0100
+++ b/tests/cloudpickle_testpkg/_cloudpickle_testpkg/mod.py     2021-09-12 00:43:12.110364370 +0100
@@ -20,19 +20,19 @@
 # module.  The following lines emulate such a behavior without being a compiled
 # extension module.

-submodule_name = '_cloudpickle_testpkg.mod.dynamic_submodule'
+submodule_name = 'tests.cloudpickle_testpkg._cloudpickle_testpkg.mod.dynamic_submodule'
 dynamic_submodule = types.ModuleType(submodule_name)

 # This line allows the dynamic_module to be imported using either one of:
-# - ``from _cloudpickle_testpkg.mod import dynamic_submodule``
-# - ``import _cloudpickle_testpkg.mod.dynamic_submodule``
+# - ``from tests.cloudpickle_testpkg._cloudpickle_testpkg.mod import dynamic_submodule``
+# - ``import tests.cloudpickle_testpkg._cloudpickle_testpkg.mod.dynamic_submodule``
 sys.modules[submodule_name] = dynamic_submodule
 # Both lines will make importlib try to get the module from sys.modules after
 # importing the parent module, before trying getattr(mod, 'dynamic_submodule'),
 # so this dynamic module could be binded to another name. This behavior is
 # demonstrated with `dynamic_submodule_two`

-submodule_name_two = '_cloudpickle_testpkg.mod.dynamic_submodule_two'
+submodule_name_two = 'tests.cloudpickle_testpkg._cloudpickle_testpkg.mod.dynamic_submodule_two'
 # Notice the inconsistent name binding, breaking attribute lookup-based import
 # attempts.
 another_submodule = types.ModuleType(submodule_name_two)
@@ -42,7 +42,7 @@
 # In this third case, the module is not added to sys.modules, and can only be
 # imported using attribute lookup-based imports.
 submodule_three = types.ModuleType(
-    '_cloudpickle_testpkg.mod.dynamic_submodule_three'
+    'tests.cloudpickle_testpkg._cloudpickle_testpkg.mod.dynamic_submodule_three'
 )
 code = """
 def f(x):
@@ -54,7 +54,7 @@
 # What about a dynamic submodule inside a dynamic submodule inside an
 # importable module?
 subsubmodule_name = (
-    '_cloudpickle_testpkg.mod.dynamic_submodule.dynamic_subsubmodule'
+    'tests.cloudpickle_testpkg._cloudpickle_testpkg.mod.dynamic_submodule.dynamic_subsubmodule'
 )
 dynamic_subsubmodule = types.ModuleType(subsubmodule_name)
 dynamic_submodule.dynamic_subsubmodule = dynamic_subsubmodule
--- a/tests/cloudpickle_test.py~        2021-09-10 11:46:35.000000000 +0100
+++ b/tests/cloudpickle_test.py 2021-09-14 09:41:21.238202700 +0100
@@ -59,7 +59,7 @@
 from .testutils import assert_run_python_script
 from .testutils import subprocess_worker

-from _cloudpickle_testpkg import relative_imports_factory
+from tests.cloudpickle_testpkg._cloudpickle_testpkg import relative_imports_factory

 _TEST_GLOBAL_VARIABLE = "default_value"
@@ -760,22 +760,22 @@
         # their parent modules are considered importable by cloudpickle.
         # See the mod_with_dynamic_submodule documentation for more
         # details of this use case.
-        import _cloudpickle_testpkg.mod.dynamic_submodule as m
+        import tests.cloudpickle_testpkg._cloudpickle_testpkg.mod.dynamic_submodule as m
         assert _should_pickle_by_reference(m)
         assert pickle_depickle(m, protocol=self.protocol) is m

         # Check for similar behavior for a module that cannot be imported by
         # attribute lookup.
-        from _cloudpickle_testpkg.mod import dynamic_submodule_two as m2
-        # Note: import _cloudpickle_testpkg.mod.dynamic_submodule_two as m2
+        from tests.cloudpickle_testpkg._cloudpickle_testpkg.mod import dynamic_submodule_two as m2
+        # Note: import tests.cloudpickle_testpkg._cloudpickle_testpkg.mod.dynamic_submodule_two as m2
         # works only for Python 3.7+
         assert _should_pickle_by_reference(m2)
         assert pickle_depickle(m2, protocol=self.protocol) is m2

         # Submodule_three is a dynamic module only importable via module lookup
         with pytest.raises(ImportError):
-            import _cloudpickle_testpkg.mod.submodule_three  # noqa
-        from _cloudpickle_testpkg.mod import submodule_three as m3
+            import tests.cloudpickle_testpkg._cloudpickle_testpkg.mod.submodule_three  # noqa
+        from tests.cloudpickle_testpkg._cloudpickle_testpkg.mod import submodule_three as m3
         assert not _should_pickle_by_reference(m3)

         # This module cannot be pickled using attribute lookup (as it does not
@@ -787,7 +787,7 @@

         # Do the same for an importable dynamic submodule inside a dynamic
         # module inside a file-backed module.
-        import _cloudpickle_testpkg.mod.dynamic_submodule.dynamic_subsubmodule as sm  # noqa
+        import tests.cloudpickle_testpkg._cloudpickle_testpkg.mod.dynamic_submodule.dynamic_subsubmodule as sm  # noqa
         assert _should_pickle_by_reference(sm)
         assert pickle_depickle(sm, protocol=self.protocol) is sm

@@ -2115,7 +2115,7 @@
     def test___reduce___returns_string(self):
         # Non regression test for objects with a __reduce__ method returning a
         # string, meaning "save by attribute using save_global"
-        from _cloudpickle_testpkg import some_singleton
+        from tests.cloudpickle_testpkg._cloudpickle_testpkg import some_singleton
         assert some_singleton.__reduce__() == "some_singleton"
         depickled_singleton = pickle_depickle(
             some_singleton, protocol=self.protocol)
@@ -2187,7 +2187,7 @@
         assert depickled_T1 is depickled_T2

     def test_pickle_importable_typevar(self):
-        from _cloudpickle_testpkg import T
+        from tests.cloudpickle_testpkg._cloudpickle_testpkg import T
         T1 = pickle_depickle(T, protocol=self.protocol)
         assert T1 is T

@@ -2514,11 +2514,11 @@
     ):
         for package_or_module in ["package", "module"]:
             if package_or_module == "package":
-                import _cloudpickle_testpkg as m
+                import tests.cloudpickle_testpkg._cloudpickle_testpkg as m
                 f = m.package_function_with_global
                 _original_global = m.global_variable
             elif package_or_module == "module":
-                import _cloudpickle_testpkg.mod as m
+                import tests.cloudpickle_testpkg._cloudpickle_testpkg.mod as m
                 f = m.module_function_with_global
                 _original_global = m.global_variable
             try:
@@ -2546,9 +2546,9 @@
         # pickled in a different way - by value and/or by reference) can
         # peacefully co-exist (e.g. without globals interaction) in a remote
         # worker.
-        import _cloudpickle_testpkg
-        from _cloudpickle_testpkg import package_function_with_global as f
-        _original_global = _cloudpickle_testpkg.global_variable
+        import tests.cloudpickle_testpkg._cloudpickle_testpkg
+        from tests.cloudpickle_testpkg._cloudpickle_testpkg import package_function_with_global as f
+        _original_global = tests.cloudpickle_testpkg._cloudpickle_testpkg.global_variable

         def _create_registry():
             _main = __import__("sys").modules["__main__"]
@@ -2568,8 +2568,8 @@
                 w.run(_create_registry)
                 w.run(_add_to_registry, f, "f_by_ref")

-                register_pickle_by_value(_cloudpickle_testpkg)
-                _cloudpickle_testpkg.global_variable = "modified global"
+                register_pickle_by_value(tests.cloudpickle_testpkg._cloudpickle_testpkg)
+                tests.cloudpickle_testpkg._cloudpickle_testpkg.global_variable = "modified global"
                 w.run(_add_to_registry, f, "f_by_val")
                 assert (
                     w.run(_call_from_registry, "f_by_ref") == _original_global
@@ -2579,10 +2579,10 @@
                 )

         finally:
-            _cloudpickle_testpkg.global_variable = _original_global
+            tests.cloudpickle_testpkg._cloudpickle_testpkg.global_variable = _original_global

-            if "_cloudpickle_testpkg" in list_registry_pickle_by_value():
-                unregister_pickle_by_value(_cloudpickle_testpkg)
+            if "tests.cloudpickle_testpkg._cloudpickle_testpkg" in list_registry_pickle_by_value():
+                unregister_pickle_by_value(tests.cloudpickle_testpkg._cloudpickle_testpkg)

     @pytest.mark.skipif(
         sys.version_info < (3, 7),
@@ -2623,12 +2623,12 @@

 def test_lookup_module_and_qualname_importable_typevar():
-    import _cloudpickle_testpkg
-    T = _cloudpickle_testpkg.T
+    import tests.cloudpickle_testpkg._cloudpickle_testpkg
+    T = tests.cloudpickle_testpkg._cloudpickle_testpkg.T
     module_and_name = _lookup_module_and_qualname(T, name=T.__name__)
     assert module_and_name is not None
     module, name = module_and_name
-    assert module is _cloudpickle_testpkg
+    assert module is tests.cloudpickle_testpkg._cloudpickle_testpkg
     assert name == 'T'

@@ -2642,8 +2642,8 @@

 def test_register_pickle_by_value():
-    import _cloudpickle_testpkg as pkg
-    import _cloudpickle_testpkg.mod as mod
+    import tests.cloudpickle_testpkg._cloudpickle_testpkg as pkg
+    import tests.cloudpickle_testpkg._cloudpickle_testpkg.mod as mod

     assert list_registry_pickle_by_value() == set()

Please let me know if you want that as PR. Here is pytest result with that patch:

+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.10.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
Using --randomly-seed=530352715
rootdir: /home/tkloczko/rpmbuild/BUILD/cloudpickle-2.0.0, configfile: tox.ini
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, freezegun-0.4.2, aspectlib-1.5.2, toolbox-0.5, rerunfailures-9.1.1, requests-mock-1.9.3, cov-2.12.1, flaky-3.7.0, benchmark-3.4.1, xdist-2.3.0, pylama-7.7.1, datadir-1.3.1, regressions-2.2.0, cases-3.6.3, xprocess-0.18.1, black-0.3.12, asyncio-0.15.1, subtests-0.5.0, isort-2.0.0, hypothesis-6.14.6, mock-3.6.1, profiling-1.7.0, randomly-3.8.0, Faker-8.12.1, nose2pytest-1.0.8, pyfakefs-4.5.1, tornado-0.8.1, twisted-1.13.3, aiohttp-0.3.0, localserver-0.5.0, anyio-3.3.1, trio-0.7.0
collected 250 items

tests/test_backward_compat.py .......
tests/cloudpickle_test.py .................................................................................................s..........................................................................................................................................
tests/cloudpickle_file_test.py .......

========================================================================= short test summary info ==========================================================================
SKIPPED [1] tests/cloudpickle_test.py:2151: Need Pickle Protocol 5 or later
===================================================================== 249 passed, 1 skipped in 23.09s ======================================================================
pytest-xprocess reminder::Be sure to terminate the started process by running 'pytest --xkill' if you have not explicitly done so in your fixture with 'xprocess.getinfo(<process_name>).terminate()'.

[pierreglaser]

But if it is only part of internal test suite it does not make to much sense to install it before testing.

I do not understand this sentence. Some cloudpickle tests are testing the pickling of constructs defined in _cloudpickle_testpkg, and such tests assume that _cloudpickle_testpkg is a pip-installed package. Thus, pip install tests/cloudpickle_testpkg must be ran before running the test suite.

Please let me know if you want that as PR.

This patch would defeat the purpose of the tests mentioned above. pip install tests/cloudpickle_testpkg should be run before running the tests. If we want this command to be ran automatically, we probably need to use toxon top of pytest to run the test suite, as tox creates isolated environment and installs cloudpickle in it before running the test suite. @kloczek I'd welcome a patch that updates cloudpickle's tox.ini file such that cloudpickle_testpkg is installed before running the tests usingpytest, see https://github.com/cloudpipe/cloudpickle/blob/master/tox.ini

[kloczek]

If we want this command to be ran automatically, we probably need to use toxon top of pytest to run the test suite, as tox

Issue is that this way automatically would you cut off from ability to use pytest strongest feature which is is running pytest against unspecified set of pytest extensions as tox will always run pytest with exact set of modules inside venv. I really do now understand why do you want to install test modules that way. I can tell you that I've never seen such approach alike with your modules. If those moduels would be needed not onlty for testing they should be installed regular way. If they are not those moduleas should be passed as pytest --pyargs <module.name> but with that code should be adapted to be used that way. Just checked my rpm packages spec fiels library and here are few examples of the moduels which are using that king of approach:

[tkloczko@barrel SPECS]$ grep %pytest *| grep -- --pyargs
python-async-generator.spec:%pytest --pyargs async_generator
python-simplejson.spec:%pytest --pyargs simplejson.tests
python-sniffio.spec:%pytest --pyargs sniffio

--pyargs <module.name> allows preload module from source tree without installing it.