bloomberg / memray

Memray is a memory profiler for Python
https://bloomberg.github.io/memray/
Apache License 2.0
13.18k stars 394 forks source link

memray table --leaks out to html can't open in browser #69

Closed lijiachang closed 2 years ago

lijiachang commented 2 years ago

python3.8, i want to known mem leaks

PYTHONMALLOC=malloc python -m memray run --output my_test.bin ./manage.py my_program then python -m memray table --leaks my_test.bin

memray give me a html, but i cant see anything in my browser,

image memray-table-my_test_51.html.zip

lijiachang commented 2 years ago

image

godlygeek commented 2 years ago

Well, it seems we somehow generated syntactically invalid Javascript, though I can't yet see how that could have happened. The generated HTML file contains (with newlines added for readability):

{"allocator": "malloc", "n_allocations": 9, "size": 360, "stack_trace": "_make_nmtuple at /app/.heroku/python/lib/python3.8/typing.py:1594", "tid": "0x7f10f5ab5080"},
{"allocator": "malloc", "n_allocations": 1, "size": 136, "stack_trace": "Info at /app/.herokuk_trace": "_path_stat at \u0026lt;frozen importlib._bootstrap_external\u0026gt;:142", "tid": "0x7f10f5ab5080"},
{"allocator": "malloc", "n_allocations": 9, "size": 1648, "stack_trace": "\u0026lt;module\u0026gt; at /app/.heroku/python/lib/python3.8/site-packages/django/db/models/functions/math.py:159", "tid": "0x7f10f5ab5080"},
{"allocator": "malloc", "n_allocations": 1, "size": 136, "stack_trace": "\u0026lt;module\u0026gt; at /app/.heroku/python/lib/python3.8/site-packages/celery/utils/serialization.py:185", "tid": "0x7f10f5ab5080"},
{"allocator": "malloc", "n_allocations": 1, "size": 75, "stack_trace": "parent at \u0026lt;frozen importlib._bootstrap\u0026gt;:393", "tid": "0x7f10f5ab5080"},
{"allocator": "malloc", "n_allocations": 3, "size": 96, "stack_trace": "max at /app/.heroku/python/lib/python3.8/site-packages/numpy/core/getlimits.py:678", "tid": "0x7f10f5ab5080"},
{"allocator": "malloc", "n_allocations": 2, "size": 642, "stack_trace": "\u0026lt;module\u0026gt; at /app/.heroku/python/lib/python3.8/site-pmalloc", "n_allocations": 1, "size": 136, "stack_trace": "\u0026lt;module\u0026gt; at /app/.heroku/python/lib/python3.8/site-packages/django/forms/models.py:106", "tid": "0x7f10f5ab5080"},

Which looks very badly wrong - the second line has a colon in a place where a colon shouldn't be, and the last line seems to have two allocations mangled together. It's as though somehow multiple threads were all trying to write to the file at once without serialization, but that doesn't make much sense.

godlygeek commented 2 years ago

I'm stumped how this possibly could have happened.

Are you able to reproduce it?

Do you still have the my_test.bin file? If so, are you able to reproduce the problem by running

python -m memray table -o different_file.html --leaks my_test.bin

or does the resulting different_file.html work correctly?

At this point, my best guess is that you managed to have two memray table runs at the same time both trying to write to the same output file, and they trampled on each other.

lijiachang commented 2 years ago

thanks for your reply, it looklike the lrzsz problem , when i use sz command transport the HTML file to my MacBook, the HTML file cant open。 when i use sftp transport HTML file,It work, I can open it。

thanks for your answer,again。 Snipaste_2022-05-05_12-46-12

godlygeek commented 2 years ago

Well, that's a fun problem! Glad you solved it.