redis / go-redis

Redis Go client
https://redis.uptrace.dev
BSD 2-Clause "Simplified" License
19.96k stars 2.36k forks source link

ZRange error "redis: can't parse array/set/push reply" and redis: invalid reply: "\r\n" #2875

Open abdullahskartal opened 8 months ago

abdullahskartal commented 8 months ago

I'm getting error while using ZRange

There are two errors coming separately at different times but the redis operation is same.

1) redis: can't parse array/set/push reply: "$24" 2) redis: invalid reply: "\r\n"

Expected Behavior

Get all sorted set items without exception or get valid error message

Current Behavior

Getting error which I don't fully understand.

Steps to Reproduce

clusterOption := &redis.ClusterOptions{
    DialTimeout:  ev.Redis.DialTimeout,
    ReadTimeout:  ev.Redis.ReadTimeout,
    WriteTimeout: ev.Redis.WriteTimeout,
    PoolSize:     ev.Redis.PoolSize,
    PoolTimeout:  ev.Redis.PoolTimeout,
    MinIdleConns: ev.Redis.MinIdleConnections,
    MaxRedirects: ev.Redis.MaxRedirects,
    Addrs:        []string{ev.Redis.Address},
    TLSConfig: &tls.Config{
        MinVersion: tls.VersionTLS12,
    },
}

redisClient := redis.NewClusterClient(clusterOption)
if err := redisClient.Ping(ctx).Err(); err != nil {
    return nil, fmt.Errorf("pinging failed, %s", err.Error())
}
ids, err := redisClient.ZRange(context.Background(), "x:123", 0, -1)

Context (Environment)

The number of elements in the sorted set is 21028 for the first error, and 475490 for the second error.

I'm using AWS MemoryDB Redis cluster.

Number of nodes: 9

Number of shards: 3

Redis version: 7.0

Golang version: 1.21.1

go-redis version: v9.3.0

rueian commented 8 months ago

The seconds error seems to be the same issue reported to rueidis https://github.com/redis/rueidis/issues/438

rueian commented 8 months ago

I tried ZRange to read all data from a zset with more than 1,000,000 members on an AWS MemoryDB Redis cluster 7.0 continuously and concurrently for hours, but I still couldn't reproduce the issue. I wonder if this issue could be a Redis bug that only happened on your specific data.

abdullahskartal commented 8 months ago

Hey @rueian,

Thats interesting. The data is just a MongoDB ids. You can take a look here. https://fakerjs.dev/api/database.html#mongodbobjectid

I have nearly 2-3 million MongoDB IDs into the sorted set and getting error ZRange. There is no special thing into the data and I didn't even use concurrency for the ZRange.

Not sure but maybe related: I'm running concurrency with ZRem after getting data with ZRange.