bloomberg / pytest-memray

pytest plugin for easy integration of memray memory profiler
https://pytest-memray.readthedocs.io/en/latest/
Apache License 2.0
335 stars 24 forks source link

Marker for fail a leaking test #45

Closed tonybaloney closed 1 year ago

tonybaloney commented 1 year ago

Feature Request

I've got some scripts using memray that run some critical code 100,000 times, I use this in the following way:

mkdir -p .profiles
PYTHONMALLOC=malloc memray run -o .profiles/memray_logrecord.py.bin -f --native memray_logrecord.py
memray flamegraph --leaks -f .profiles/memray_logrecord.py.bin
python -m http.server --directory .profiles 8000

Then I manually look at the HTML report for leaky lines and raise bugs against them. Is there any way to automate this using this pytest package?

The test script is just something like:

def do():
    # setup... 

    for _ in range(100_000):
        ... # do code things

    # assert

if __name__ == "__main__":
    do()
gaborbernat commented 1 year ago

Any reason the https://pytest-memray.readthedocs.io/en/latest/usage.html#limit-memory does not help?

seebi commented 1 year ago

@gaborbernat thanks for the link. Is there a way to set a default limit for all tests (without the need for adding this to all tests manually) - something like a memory base fence ...?

gaborbernat commented 1 year ago

There is not at the moment, but PR welcome. And you can also set it to your project via a pytest hook.

seebi commented 1 year ago

Do you have a link to an example for this as well ? :-)

sarahmonod commented 1 year ago

You can use any of those hooks: https://docs.pytest.org/en/7.1.x/reference/reference.html

But I would personally recommend using an autouse fixture instead: https://docs.pytest.org/en/6.2.x/fixture.html#autouse-fixtures-fixtures-you-don-t-have-to-request

pablogsal commented 1 year ago

I'm adding a new marker in #52 for checking memory leaks. Please @tonybaloney check if this would work for your use case.