approvals / ApprovalTests.Python

ApprovalTests for python
Apache License 2.0
141 stars 48 forks source link

Support `__test__` attribute in PyTest test discovery #161

Open James-Ansley opened 2 months ago

James-Ansley commented 2 months ago

A recent release included support for alternative (i.e. non-"test") test prefixes.

Pytest, also uses a custom __test__ dunder when discovering tests: python.py#L349. I.e. any function with the attribute __test__ = True is considered a test regardless of its prefix. Currently, this library does not support this dunder.

Would it be possible to support the __test__ dunder in the recently added is_pytest_test method?

I am happy to work on this unless someone else sees a quick fix.

Suggested Fix

An additional test in the StackFrameNamer class for the __test__ dunder could be something like the following:

@staticmethod
def _is_marked_with_test_dunder(
      method_name: str,
      frame_globals: dict[str, "Any"],
) -> bool:
    function = frame_globals.get(method_name)
    return (
          function is not None
          and hasattr(function, "__test__")
          and getattr(function, "__test__") is True  # i.e. True not truthy
    )

@staticmethod
def is_pytest_test(frame: FrameInfo) -> bool:
    method_name = frame[3]
    frame_globals = frame.frame.f_globals
    patterns = PytestConfig.test_naming_patterns
    return (
          StackFrameNamer._is_match_for_pytest(method_name, patterns)
          or StackFrameNamer._is_marked_with_test_dunder(
                method_name,
                frame_globals,
          )
    )

Here, the function name is looked up in the frame globals, and if found, it checks for the __test__ dunder. I would need to refresh my memory on Python's FrameInfo to check whether this is the best way of doing this.

nitsanavni commented 2 months ago

Looks good! Usually I would invite you to the weekly ensemble, but it's not NZ timezone friendly unfortunately. Anyway, know that you're always welcome there - https://github.com/approvals/ApprovalTests.Python/blob/main/docs/Contribute.md.