StackExchange / StackExchange.Redis

General purpose redis client
https://stackexchange.github.io/StackExchange.Redis/
Other
5.85k stars 1.5k forks source link

Unexpected end of Steam #2643

Closed STRATZ-Ken closed 5 months ago

STRATZ-Ken commented 5 months ago

Hello,

ServiceStack Redis version 8.1.2.

I am currently using a Redis instance for Pub/Sub where I am setting fairly large blob objects. These objects between all of the messages its around 50Mbit/sec. The error below is causing an issue where it will reconnect each time. However, once in a while the reconnect will not happen and the entire thread comes to a halt. No additional errors will be reported, and no reconnection. I would love to solve the actual timeout issue, maybe I am missing a config line? I found config get client-output-buffer-limit but I ran some tests on dd if=/dev/zero bs=$((1024*1024)) count=51 | redis-cli -h 1.2.3.4 -p 1234 -a 'XXXXX' -x PUBLISH myStream and it didn't seem to have an issue pushing that size file. Please let me know if I left anything out.

Error :

10:37:48|ERROR|Recieved onStop
10:37:48|ERROR|Recieved onError ServiceStack.Redis.RedisResponseException: Unexpected end of Stream
   at ServiceStack.Redis.RedisNativeClient.ParseSingleLine(String r) in /home/runner/work/ServiceStack/ServiceStack/ServiceStack.Redis/src/ServiceStack.Redis/RedisNativeClient_Utils.cs:line 1033
   at ServiceStack.Redis.RedisNativeClient.ReadMultiData() in /home/runner/work/ServiceStack/ServiceStack/ServiceStack.Redis/src/ServiceStack.Redis/RedisNativeClient_Utils.cs:line 1092
   at ServiceStack.Redis.RedisSubscription.SubscribeToChannels(String[] channels) in /home/runner/work/ServiceStack/ServiceStack/ServiceStack.Redis/src/ServiceStack.Redis/RedisSubscription.cs:line 43
   at ServiceStack.Redis.RedisPubSubServer.RunLoop() in /home/runner/work/ServiceStack/ServiceStack/ServiceStack.Redis/src/ServiceStack.Redis/RedisPubSubServer.cs:line 336
10:28:12|ERROR|Recieved onStop
10:28:12|ERROR|Recieved onError ServiceStack.Redis.RedisException: [18:28:12.445] Unable to Connect: sPort: 64103, Error: Cannot access a disposed object.
Object name: 'System.Net.Sockets.Socket'.
   at System.Net.Sockets.Socket.ThrowObjectDisposedException()
   at System.Net.Sockets.Socket.Send(IList`1 buffers, SocketFlags socketFlags, SocketError& errorCode)
   at System.Net.Sockets.Socket.Send(IList`1 buffers, SocketFlags socketFlags)
   at ServiceStack.Redis.RedisNativeClient.FlushSendBuffer() in /home/runner/work/ServiceStack/ServiceStack/ServiceStack.Redis/src/ServiceStack.Redis/RedisNativeClient_Utils.cs:line 521
   at ServiceStack.Redis.RedisNativeClient.SendReceive[T](Byte[][] cmdWithBinaryArgs, Func`1 fn, Action`1 completePipelineFn, Boolean sendWithoutRead, String operation) in /home/runner/work/ServiceStack/ServiceStack/ServiceStack.Redis/src/ServiceStack.Redis/RedisNativeClient_Utils.cs:line 659
 ---> System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Net.Sockets.Socket'.
   at System.Net.Sockets.Socket.ThrowObjectDisposedException()
   at System.Net.Sockets.Socket.Send(IList`1 buffers, SocketFlags socketFlags, SocketError& errorCode)
   at System.Net.Sockets.Socket.Send(IList`1 buffers, SocketFlags socketFlags)
   at ServiceStack.Redis.RedisNativeClient.FlushSendBuffer() in /home/runner/work/ServiceStack/ServiceStack/ServiceStack.Redis/src/ServiceStack.Redis/RedisNativeClient_Utils.cs:line 521
   at ServiceStack.Redis.RedisNativeClient.SendReceive[T](Byte[][] cmdWithBinaryArgs, Func`1 fn, Action`1 completePipelineFn, Boolean sendWithoutRead, String operation) in /home/runner/work/ServiceStack/ServiceStack/ServiceStack.Redis/src/ServiceStack.Redis/RedisNativeClient_Utils.cs:line 659
   --- End of inner exception stack trace ---
   at ServiceStack.Redis.RedisNativeClient.CreateConnectionError(Exception originalEx) in /home/runner/work/ServiceStack/ServiceStack/ServiceStack.Redis/src/ServiceStack.Redis/RedisNativeClient_Utils.cs:line 371
   at ServiceStack.Redis.RedisNativeClient.SendReceive[T](Byte[][] cmdWithBinaryArgs, Func`1 fn, Action`1 completePipelineFn, Boolean sendWithoutRead, String operation) in /home/runner/work/ServiceStack/ServiceStack/ServiceStack.Redis/src/ServiceStack.Redis/RedisNativeClient_Utils.cs:line 703
   at ServiceStack.Redis.RedisNativeClient.SendExpectMultiData(Byte[][] cmdWithBinaryArgs) in /home/runner/work/ServiceStack/ServiceStack/ServiceStack.Redis/src/ServiceStack.Redis/RedisNativeClient_Utils.cs:line 801
   at ServiceStack.Redis.RedisNativeClient.UnSubscribe(String[] fromChannels) in /home/runner/work/ServiceStack/ServiceStack/ServiceStack.Redis/src/ServiceStack.Redis/RedisNativeClient.cs:line 2167
   at ServiceStack.Redis.RedisSubscription.UnSubscribeFromAllChannels() in /home/runner/work/ServiceStack/ServiceStack/ServiceStack.Redis/src/ServiceStack.Redis/RedisSubscription.cs:line 116
   at ServiceStack.Redis.RedisSubscription.Dispose() in /home/runner/work/ServiceStack/ServiceStack/ServiceStack.Redis/src/ServiceStack.Redis/RedisSubscription.cs:line 152
   at ServiceStack.Redis.RedisPubSubServer.RunLoop() in /home/runner/work/ServiceStack/ServiceStack/ServiceStack.Redis/src/ServiceStack.Redis/RedisPubSubServer.cs:line 338

Code

var redisPubSubCache = new RedisPubSubServer(RedisRepository.Redis, new string[] { "BetCache", "Reporter", "GameLiveData" })
{
    AutoRestart = true,
    HeartbeatInterval = TimeSpan.FromSeconds(5),
    WaitBeforeNextRestart = TimeSpan.FromSeconds(10),
    OnMessage = (channel, msg) =>
    {
        switch (channel)
        {
            case "BetCache":
                {
                    break;
                }
            case "GameLiveData":
                {
                    break;
                }
            case "Reporter":
                {
                    break;
                }
        }
    },
    OnStop = () =>
    {
        Logger.Error("Recieved onStop");
    },
    OnError = (error) =>
    {
        Logger.Error("Recieved onError " + error);
    }
}.Start();
NickCraver commented 5 months ago

Hey there! ServiceStack.Redis is not StackExchange.Redis, that's another library/repo located here: https://github.com/ServiceStack/ServiceStack