bloomberg / pytest-memray

pytest plugin for easy integration of memray memory profiler
https://pytest-memray.readthedocs.io/en/latest/
Apache License 2.0
345 stars 24 forks source link

Empty report when running with pytest-xdist #2

Closed orsinium closed 1 year ago

orsinium commented 2 years ago

Bug Report

Current Behavior The plugin produces an empty output when running tests with pytest-xdist.

Input Code

a = []

def something():
    for i in range(10_000):
        a.append(i)

def test_something():
    something()

Running multiple processes:

image

Running without distribution:

image

Environment

Possible-solutions: For the beginning, it would be sufficient to detect running tests with pytest-xdist and warn the user that -n0 is required to make pytest-memray work. Of course, moving forward, it would be great to make both plugins work together. Running tests in a single process is too hardcore for some projects.

petr-tik commented 2 years ago

i am thinking about adding a test and minimal implementation to document the absence of integration between xdist and memray. What is better: return a ExitCode.USAGE_ERROR OR print a warning (warnings.warn kind of warning?) to the terminal and ask the user to pass -n1, if they want to use memray?

gaborbernat commented 2 years ago

Is there a way we can instead add support for xdist?

pablogsal commented 2 years ago

Is there a way we can instead add support for xdist?

Yes, but is not trivial. The problem is that currently the information is collected by every worker but the process that is orchestrating the build is the want printing of the summary and when is going done it finds no information because the workers are not sending the information back. Another problem is that we should not leave the result files lying around until the full test suite finishes because it may be a lot of data (currently we delete the files after every test finishes so they never pile up).

gaborbernat commented 2 years ago

I'm fine with displaying a usage error for now and nudge to -n1. Just would like another issue open that handles support for that.

petr-tik commented 2 years ago

I'm fine with displaying a usage error for now and nudge to -n1

I will add a test and code for the ExitCode.UsageError then

orsinium commented 2 years ago

I'm fine with displaying a usage error for now and nudge to -n1

A minor correction: it should be -n0. AFAIK, -n1 will run with the distribution but with just one worker. -n0 will disable pytest-xdist altogether.