pytest-dev / pytest

The pytest framework makes it easy to write small tests, yet scales to support complex functional testing
https://pytest.org
MIT License
12.02k stars 2.67k forks source link

Pytest coverage seems to not work when breaking a while True code via multiprocessing terminate. #7062

Closed newskooler closed 4 years ago

newskooler commented 4 years ago

I have an async python function which connects to a websocket and then uses a while True loop to collect and save the information form the websocket to a DB.

The function looks very similar to this one (source);:

async def get_data(insert_function):
    bitonic_address = "wss://api.bl3p.eu/1/BTCUSD/trades"
    async with websockets.connect(bitonic_address) as websocket:
        while True:
            message_str = await websocket.recv()
            message = json.loads(message_str)
            await insert_function(message)

def run():
    asyncio.run(get_data(insert_function))

I was thinking of way to test it and tried getting some help from Stack Overflow (you can see my question here). The way I figured out to test whether the function connects and writes correctly the information is by running the. while loop for a limited time and then checking the results and cleaning afterwards.

I achieve the time out by running the function via a multiprocessor. Afterwards, following a bunch of asserts, I evaluate if the results are correct.

def test_a_while_loop():
    # Start through multiprocessing in order to have a timeout.
    p = multiprocessing.Process(
        target= run
        name="Foo",
    )
    try:
        p.start()
        # my timeout
        time.sleep(10)
        p.terminate()
    finally:
        # Cleanup.
        p.join()

    # Asserts below
    ...

All of the above works fine, however when I look at the coverage report, it wrongly states the much of the lines of code are not covered.

Is this a bug or is it because I time out the while True loop through a multiprocessor?

How can I fix this?

The-Compiler commented 4 years ago

pytest has nothing to do with coverage. If you run Python code in another process and want it to be covered, you will need to set coverage.py up correctly or use the pytest-cov plugin.

Closing this now as I don't think there's anything actionable for pytest here, but feel free to follow up if needed!

newskooler commented 4 years ago

Well this gives me a good lead. Thanks.