RedisCloud doesn't permit EVALSHA

adamlwgriffiths commented 8 years ago

When trying to run on RedisCloud, I get the below error. The error makes not of using EVAL instead of EVALSHA. I'm not familiar with the lua capabilities, nor the internals of RMA, so I'm not sure if it is feasible to fall back to using EVAL if EVALSHA fails?

Any suggestions?

rma -s <host> -p <port> -d 0 -a <password>
Match *:   0%|                                           | 0/331 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/redis/client.py", line 2694, in __call__
    return client.evalsha(self.sha, len(keys), *args)
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/redis/client.py", line 1944, in evalsha
    return self.execute_command('EVALSHA', sha, numkeys, *keys_and_args)
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/redis/client.py", line 573, in execute_command
    return self.parse_response(connection, command_name, **options)
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/redis/client.py", line 585, in parse_response
    response = connection.read_response()
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/redis/connection.py", line 582, in read_response
    raise response
redis.exceptions.NoScriptError: No matching script. Please use EVAL.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/adamgriffiths/.anaconda/envs/systema3/bin/rma", line 11, in <module>
    sys.exit(main())
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/rma/cli/rma_cli.py", line 92, in main
    app.run()
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/rma/application.py", line 106, in run
    for v in scanner.scan(limit=self.limit):
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/rma/scanner.py", line 85, in scan
    for key_tuple in self.batch_scan():
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/rma/scanner.py", line 53, in batch_scan
    yield from self.resolve_types(ret)
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/rma/scanner.py", line 61, in resolve_types
    raise e
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/rma/scanner.py", line 58, in resolve_types
    key_with_types = msgpack.unpackb(self.resolve_types_script(ret))
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/redis/client.py", line 2699, in __call__
    return client.evalsha(self.sha, len(keys), *args)
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/redis/client.py", line 1944, in evalsha
    return self.execute_command('EVALSHA', sha, numkeys, *keys_and_args)
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/redis/client.py", line 573, in execute_command
    return self.parse_response(connection, command_name, **options)
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/redis/client.py", line 585, in parse_response
    response = connection.read_response()
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/redis/connection.py", line 582, in read_response
    raise response
redis.exceptions.ResponseError: Error running script (call to f_b89215e5d8808de13e1c715aaedbb744ce4536e7): @user_script:5: @user_script: 5: This Redis command is not allowed from scripts

misterion commented 8 years ago

Try this as fast workaround (in scanner.py line 28):

self.pipeline_mode = True

Now we switch to pipeline mode only on CROSSSLOT error. Looks like we should do this withNoScriptError` error too.

adamlwgriffiths commented 8 years ago

New error trace with self.pipeline_mode = True

Match *:   0%|                                           | 0/326 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/adamgriffiths/.anaconda/envs/systema3/bin/rma", line 9, in <module>
    load_entry_point('rma', 'console_scripts', 'rma')()
  File "/media/sf_shared/Workspace/systema/systema-profiler/redis-memory-analyzer/rma/cli/rma_cli.py", line 92, in main
    app.run()
  File "/media/sf_shared/Workspace/systema/systema-profiler/redis-memory-analyzer/rma/application.py", line 106, in run
    for v in scanner.scan(limit=self.limit):
  File "/media/sf_shared/Workspace/systema/systema-profiler/redis-memory-analyzer/rma/scanner.py", line 86, in scan
    for key_tuple in self.batch_scan():
  File "/media/sf_shared/Workspace/systema/systema-profiler/redis-memory-analyzer/rma/scanner.py", line 54, in batch_scan
    yield from self.resolve_types(ret)
  File "/media/sf_shared/Workspace/systema/systema-profiler/redis-memory-analyzer/rma/scanner.py", line 66, in resolve_types
    key_with_types = self.resolve_with_pipe(ret)
  File "/media/sf_shared/Workspace/systema/systema-profiler/redis-memory-analyzer/rma/scanner.py", line 78, in resolve_with_pipe
    key_with_types = [{'type': x, 'encoding': y} for x, y in chunker(pipe.execute(), 2)]
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/redis/client.py", line 2626, in execute
    return execute(conn, stack, raise_on_error)
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/redis/client.py", line 2567, in _execute_pipeline
    self.raise_first_error(commands, response)
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/redis/client.py", line 2574, in raise_first_error
    raise r
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/redis/client.py", line 2562, in _execute_pipeline
    self.parse_response(connection, args[0], **options))
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/redis/client.py", line 2584, in parse_response
    self, connection, command_name, **options)
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/redis/client.py", line 585, in parse_response
    response = connection.read_response()
  File "/home/adamgriffiths/.anaconda/envs/systema3/lib/python3.5/site-packages/redis/connection.py", line 582, in read_response
    raise response
redis.exceptions.ResponseError: Command # 2 (OBJECT ENCODING b'rq:job:0071800e-0908-48af-9e0a-59a2e21cba20') of pipeline caused error: command 'OBJECT' is not allowed

Perhaps RedisLabs is just too strict/paranoid to run this code?

misterion commented 8 years ago

I can understand why EVAL may be disabled but OBJECT looks like much of paranoia. I am afraid this tool cant work without OBJECT ENCODING support :( Please, could you check isDEBUG OBJECTcommand is allowed? If yes, i can write some fallback logic to use it instead ofOBJECT`.

adamlwgriffiths commented 8 years ago

How would I go about that? I haven't used Redis in depth, only for key/value, no Lua scripting.

I've written a script to manually iterate keys and get info, it's not great but it does what I need for now. As such, I don't need this fixed (although it would be nice). I'm happy to run some code if you want me to, but I also don't want to take up your time chasing down an issue that isn't really yours, it's RedisLabs. I don't know if you just want to close the issue. I'll leave it up to you.

misterion commented 8 years ago

The issue with EVAL should be fixed, so tnx for it anyway. Can you run in redis cli command 'DEBUG OBJECT anyKeyYouWant' ?

adamlwgriffiths commented 8 years ago

> DEBUG OBJECT rma_test_key
(error) ERR command 'DEBUG OBJECT' is not allowed

Looks like RedisLabs is locked down big time =(

However, thanks for your time looking into this, its appreciated =)

gamenet / redis-memory-analyzer

RedisCloud doesn't permit EVALSHA #25