darrenburns / ward

Ward is a modern test framework for Python with a focus on productivity and readability.
https://ward.readthedocs.io
MIT License
1.21k stars 53 forks source link

Question: How do you assert the output? #351

Open mgzenitech opened 2 years ago

mgzenitech commented 2 years ago

I've tried to set up my own fixture for capturing the output:

import sys
from contextlib import contextmanager
from io import StringIO
from typing import Callable, Iterable
from ward import fixture, Scope

@fixture(scope = Scope.Global)
def capture_output() -> Callable: # pylint: disable=unused-variable
    @contextmanager
    def __capture_output() -> Iterable[StringIO]:
        sys.stdout = StringIO()
        try:
            yield sys.stdout
        finally:
            sys.stdout = sys.__stdout__
    return __capture_output

But for this breaks the capture-output functionality of Ward. Is there a standard way of asserting the stdout + stderr in tests and not breaking anything?

Screenshot from 2022-10-24 14-29-21

darrenburns commented 2 years ago

This is a bit of a tricky area. At the point where the fixture executes, hasn't sys.stdout already been replaced with something else by Ward? If you print it out you might see that it has.

An idea that might work is this:

Inside your fixture, you could re-assign sys.stdout to some proxy file-like object which captures the output and then forwards it on to whatever was in sys.stdout before. Something like sys.stdout = CapturingProxy(sys.stdout).

CapturingProxy would be a file-like object implementing read/write etc, and storing everything that it proxies through to the original stdout. Then, you'd return the content that the CapturingProxy holds.

The scope of the fixture would probably need to be Test as well for this to work, so that each test only records what was written during its own execution.

mgzenitech commented 2 years ago

Hm... Tried https://pypi.org/project/capturer/ but got an error with UnsupportedOperation: fileno as Ward mutates some low level things I think.

Update, probably due to this: Screenshot from 2022-10-24 18-18-40

Same issue as reported here: https://github.com/darrenburns/ward/issues/339

AndydeCleyre commented 2 years ago

FWIW here's a hacky setup I'm using. Instead of calling the functions directly in the tests, I wrap them with a utility function that temporarily sets stdout to a StringIO, and returns the output as str.