NO TESTS RAN failure on Noble for an ament_python package

[stonier]

Hi all,

I'm seeing a difference in behavior between Iron and Jammy for ROS PR jobs using ament_python.

• Package: py_trees
• PR with CI Jobs: PR 450
• :heavy_check_mark: Iron PR Job - Console Log - Succeeded, despite Ran 0 tests
• :x: Rolling PR Job - Console Log - Failed on NO TESTS RAN

Tests are covered by github actions, so I've been leaving them out on the ROS PR Job (which serves mainly to smoke test deb builds). Has there been a toggle in colcon or the underlying tools in Noble that now require them?

I'm actually surprised it's doing anything here at all since there is no callout to pytest in [setup.py].

[cottsay]

Hi there.

There are two strategies that colcon uses to test Python packages. The default is to directly invoke the unittest module. If colcon detects that a package has a test dependency on pytest, it will use pytest instead.

So your package uses unittest. It appears that there was a change in behavior in Python 3.12: python/cpython#102051

I'm open to suggestions here, but I'm guessing that from your perspective you'd like the pre-3.12 behavior of "no error" when there were no tests. Is that correct?

[stonier]

there was a change in behavior in Python 3.12: https://github.com/python/cpython/pull/102051

Oh, good digging. Thanks. I was barking up the colcon/pytest tree.

So your package uses unittest.

Is it defaulting to that? I don't specify it anywhere.

I'm open to suggestions here, but I'm guessing that from your perspective you'd like the pre-3.12 behavior of "no error" when there were no tests. Is that correct?

I'm not sure yet. It would be reasonable to expect that if no test configuration is provided, no tests are run. However, this is all against my usual bias for requiring tests everywhere. Might be worth just activating the pytests that are there and see if the extra configuration is tolerable. Let me explore for a bit.

[cottsay]

Is it defaulting to that? I don't specify it anywhere.

That's correct. That was chosen as the default likely because it's built into Python's standard library and will reliably be available.

For a low-effort fix, it might be good to add a super simple unittest-compatible import check. Very high bang-for-your-buck minimal test.

import unittest

class ImportTest(unittest.TestCase):

    def test_import(self):
        import py_trees  # noqa: F401

[stonier]

Aye, was thinking along those lines too. Thanks!