gamenet / redis-memory-analyzer

Redis memory profiler to find the RAM bottlenecks throw scaning key space in real time and aggregate RAM usage statistic by patterns.
MIT License
772 stars 80 forks source link

Fails with TypeError if a key containing integer was removed #22

Closed drdaeman closed 8 years ago

drdaeman commented 8 years ago

First of all, thank you for this tool.

I've encountered an issue while I was trying to analyze what's taking the memory. While there's a guard against some cases when keys were removed after the scan, rma crashes when it encounters an integer-containing key that's already gone when ValueString gets to it.

Here's an example traceback:

Traceback (most recent call last):
  File ".../rma", line 9, in <module>
    load_entry_point('rma==0.1.11', 'console_scripts', 'rma')()
  File ".../rma/cli/rma_cli.py", line 92, in main
    app.run()
  File ".../rma/application.py", line 118, in run
    str_res += self.do_ram(keys)
  File ".../rma/application.py", line 153, in do_ram
    ret += (rule.analyze(keys=aggregate_patterns, total=total_keys))
  File ".../rma/rule/ValueString.py", line 81, in analyze
    with RealStringEntry(redis=self.redis, info=key_info, use_debug=use_debug_command) as stat:
  File ".../rma/rule/ValueString.py", line 34, in __init__
    self.useful_bytes = self.get_int_encoded_bytes(redis, key_name)
  File ".../rma/rule/ValueString.py", line 13, in get_int_encoded_bytes
    num_value = int(redis.get(key_name))
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

The solution seems to handle TypeError as well as ValueError in RealStringEntry.get_int_encoded_bytes. E.g. replacing except ValueError: with except (ValueError, TypeError):. Tried this on my local copy and it worked for me - I got a sensible-looking report. I'm not really sure if that's completely correct, though - maybe manually raising ResponseError("no such key") would be a better approach there.

Thanks!

misterion commented 8 years ago

Tnx, your solution seems right. I'l check and fix it this monday.

misterion commented 8 years ago

Fixed in 0.1.12

fenelon commented 8 years ago

Am I getting the same error on 0.1.12? (cleared cache, installed from github)

Apply rules Processing String patterns: 42%|██████████████████████████████████████████████████▊ | 420157/992681 [00:32<00:43, 13024.41it/s]Traceback (most recent call last): File "/usr/local/bin/rma", line 9, in load_entry_point('rma==0.1.12', 'console_scripts', 'rma')() File "/usr/local/lib/python3.5/site-packages/rma-0.1.12-py3.5.egg/rma/cli/rma_cli.py", line 92, in main File "/usr/local/lib/python3.5/site-packages/rma-0.1.12-py3.5.egg/rma/application.py", line 118, in run File "/usr/local/lib/python3.5/site-packages/rma-0.1.12-py3.5.egg/rma/application.py", line 153, in do_ram File "/usr/local/lib/python3.5/site-packages/rma-0.1.12-py3.5.egg/rma/rule/ValueString.py", line 83, in analyze File "/usr/local/lib/python3.5/site-packages/rma-0.1.12-py3.5.egg/rma/rule/ValueString.py", line 36, in init File "/usr/local/lib/python3.5/site-packages/rma-0.1.12-py3.5.egg/rma/rule/ValueString.py", line 13, in get_int_encoded_bytes TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

misterion commented 8 years ago

Sorry for that, missing more one place with this error. Commit fix with 0.1.13