ValueError: invalid literal for int() with base 10: b'OK'

[ninoseki]

Hello, thanks for creating & maitaning this great library. Recently have ValuError issue occasionally. So Let me report.

Environment

• Python: 3.11
• arq: 0.25.0
• hiredis: 2.2.1
• redis: 4.4.2

Error

Not always but sometimes the following ValueError happens

ValueError: invalid literal for int() with base 10: b'OK'
  File "arq", line 8, in <module>
    sys.exit(cli())
  File "click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "arq/cli.py", line 54, in cli
    run_worker(worker_settings_, **kwargs)
  File "arq/worker.py", line 868, in run_worker
    worker.run()
  File "arq/worker.py", line 306, in run
    self.loop.run_until_complete(self.close())
  File "asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
  File "arq/worker.py", line 843, in close
    await self.pool.delete(self.health_check_key)
  File "redis/asyncio/client.py", line 505, in execute_command
    return await conn.retry.call_with_retry(
  File "redis/asyncio/retry.py", line 59, in call_with_retry
    return await do()
  File "redis/asyncio/client.py", line 481, in _send_command_parse_response
    return await self.parse_response(conn, command_name, **options)
  File "redis/asyncio/client.py", line 536, in parse_response
    retval = self.response_callbacks[command_name](response, **options)

This error happens when executing DEL command.

[samuelcolvin]

Are you sure this is an issue with arq? Looks like it could be the redis package causing the error.

[ninoseki]

To be honest, I'm not sure. So sorry if it is a false flag. I try to reproduce the issue with redis & hiredis but I cannot. So I guess that it may be an issue on arq side.

[samuelcolvin]

I think worth reporting on the redis-py project - it's pretty deep in their code.

[ninoseki]

Ok thanks I will do that. Sorry for taking your time.

[samuelcolvin]

No problem, happy to leave this open and you can link to this issue when you create one on redis-py.

[JonasKs]

Did you create an issue in redis-py?😊

[ninoseki]

Sorry for being late/lazy. I try to create an issue with reproducible steps. But I'm failing for that. (Maybe it is an issue on a Redis server I use)

[ninoseki]

Hmm, I cannot reproduce the issue with the official Reids container image. So I'd like to conclude that this is a false flag. Sorry for taking your time. I have to investigate more before submitting the issue. My bad.

As a note, I think this is an issue on a Redis server I use, and I applied a dirty patch to mitigate the issue.

from typing import cast

from arq import ArqRedis
from redis.utils import str_if_bytes

def int_or_ok(response: int | str | bytes) -> int:
    try:
        return int(response)
    except ValueError:
        pass

    if str_if_bytes(response) == "OK":
        return 1

    return 0

async def startup(ctx: dict) -> None:
    redis = cast(ArqRedis, ctx["redis"])
    redis.set_response_callback("DEL", int_or_ok)

[JonasKs]

No worries, thanks for reporting back 😊