Open pquentin opened 7 years ago
Same happens for me.. I'm on memory_profiler-0.52.0
Same here, also on 0.52.0 edit1: seems to be somehow specific to functions that create child processes. edit2: executing with the -m memory_profiler flag works.
I gladly accept pull requests :-)
Could you please you provide a short, self-contained, correct code snippet that showcases the issue so we can try to reproduce it?
still failing in 0.54.0
@mklosi can you please give us an example to reproduce this, as per my previous comment? I am sorry to say that "still failing" kind of messages are not helpful for us, so it's unlikely that we will be able to fix the issue without more information.
When trying to run mprof run
command, I'm getting the same error:
Traceback (most recent call last): File "/Users/pedro.guridi/.virtualenvs/processing_engine/bin/mprof", line 10, in <module> sys.exit(main()) File "/Users/pedro.guridi/.virtualenvs/processing_engine/lib/python3.7/site-packages/mprof.py", line 592, in main actions[get_action()]() File "/Users/pedro.guridi/.virtualenvs/processing_engine/lib/python3.7/site-packages/mprof.py", line 242, in run_action multiprocess=args.multiprocess, stream=f) File "/Users/pedro.guridi/.virtualenvs/processing_engine/lib/python3.7/site-packages/memory_profiler.py", line 363, in memory_usage stream.write("MEM {0:.6f} {1:.4f}\n".format(*mem_usage)) TypeError: format() argument after * must be an iterable, not NoneType
The command I'm using is: mprof run python my_script.py
if __name__ == "__main__":
entry point, and from there I'm executing my functions. Update here, I have a precise example to reproduce the crash:
running: mprof run test_fails.py
:
NLP = spacy.load("en_core_web_lg", disable=["parser", "tagger"])
if __name__ == "__main__":
print("This doesn't work")
mprof run python test_fails.py
mprof: Sampling memory every 0.1s
This doesn't work
Traceback (most recent call last):
File "/Users/pedro.guridi/.virtualenvs/processing_engine/bin/mprof", line 10, in <module>
sys.exit(main())
File "/Users/pedro.guridi/.virtualenvs/processing_engine/lib/python3.7/site-packages/mprof.py", line 592, in main
actions[get_action()]()
File "/Users/pedro.guridi/.virtualenvs/processing_engine/lib/python3.7/site-packages/mprof.py", line 242, in run_action
multiprocess=args.multiprocess, stream=f)
File "/Users/pedro.guridi/.virtualenvs/processing_engine/lib/python3.7/site-packages/memory_profiler.py", line 363, in memory_usage
stream.write("MEM {0:.6f} {1:.4f}\n".format(*mem_usage))
TypeError: format() argument after * must be an iterable, not NoneType
To reproduce this error, install Spacy using pip, and add the data model with: python -m spacy download en_core_web_lg
Thanks @pguridi, that's useful. We'll try to give this a look soon.
Just wanted to chime in and say I ran across the same problem and subsequently found this discussion. I'm using version 0.55.0 on macOS (10.14.5). There may be other reasons as well, but one cause of this error stems from permissions. Running with sudo
solved the issue for me. What happens is that the psutil.AccessDenied
caught here causes _get_memory
to return a None
, which then cannot be unpacked in the stream.write
call here. Not sure what solution you all would prefer, but two ideas are,
1) Return a list of dummy values (e.g. [0.0, 0.0]
) instead of passing
2) Propagate the error and potentially also log a warning
mprof run
occassionally fails with a TypeError. It happens, say one out of ten times. I'm on macOS and use memory_profiler 0.47 with psutil 5.3.1. Here's my traceback:I don't have a perfect reproducer but
_get_memory
can indeed return None with psutil onpsutil.AccessDenied
.