search

pythonprofilers / memory_profiler

Monitor Memory usage of Python code
http://pypi.python.org/pypi/memory_profiler
Other
4.39k stars 380 forks source link

memory_profiler vs valgrind #296

Open jaytimbadia opened 4 years ago

jaytimbadia commented 4 years ago

I am using memory_profiler to check json parse usage, along with valgrind tool also to check the same. But I am getting different results

Line # Mem usage Increment Occurences Line Contents

29   36.188 MiB   36.188 MiB           1   @profile
30                                                         def load_json(filename):
31   36.188 MiB    0.000 MiB           1       with open(filename) as fh:
32   48.203 MiB   12.016 MiB           1           data = json.load(fh)

Using Valgrind

MEMORY USAGE: marshal: 73.8 Mb cbor: 75.6 Mb msgpack: 75.8 Mb pickle: 76.3 Mb json: 78.9 Mb jsonlib: 92.0 Mb ujson: 92.1 Mb rapidjson: 100.1 Mb

I am using a json file of 12MB to parse.

I mean how they both can give different results. Also http://explique.me/Ijson/ article has used 12MB file to parse using memory_profiler stats chech and getting 78MB but why I am getting 12 MB only in memory profiler? Which one is correct? Please let me know, if any questions.