NodeRedis / node-redis-parser

A high performance Redis protocol (RESP) parser for JavaScript. Used by Node Redis & ioredis.
MIT License
88 stars 36 forks source link

Protocol error, got "\"" as reply type byte #11

Closed timcosta closed 7 years ago

timcosta commented 8 years ago
ReplyError: Protocol error, got "\"" as reply type byte. Please report this.
    at parseType (/my-directory/node_modules/redis-parser/lib/parser.js:224:34)
    at JavascriptRedisParser.execute (/my-directory/node_modules/redis-parser/lib/parser.js:404:20)

Redis version 3.2.0

BridgeAR commented 8 years ago

You use the newest parser version? And do you have any more information about what exactly you did?

arjenvanderende commented 8 years ago

On our production environment we see the same error, a couple of times per day.

Protocol error, got "0" as reply type byte. Please report this.   

From our application logs, it appears this error this thrown directly after reconnecting when the connection to Redis was lost.

14:00:23.000 Redis connection to redis-1 lost for 0ms (retrycount: 1)
14:00:24.000 Redis connections are healthy.
14:00:24.000 uncaughtException: Protocol error, got "0" as reply type byte. Please report this.

From the examples of node_redis I see that it's preferable to wait for the ready event of the client before sending any commands. We don't (yet) do that. Could that be part of the issue?

BridgeAR commented 8 years ago

@arjenvanderende the ready event should not be of any concern in this case. Would you be so kind and activate the debug_mode? That way I might have at least some basic information to start debugging this as I have no starting point currently. And please update the parser to v.2.0.4 as 2.0.3 might return corrupted multi byte characters from time to time.

arjenvanderende commented 8 years ago

@BridgeAR I'm hesitant to enable debug mode on our production environment. The docs discourage using debug mode in production because of the performance penalty. There's also no indication of how long we'd need to have debug mode enabled, because on some days the protocol error happens around 20 times, while on other days it doesn't happen at all.

A point of interest is that we have three application in production, but we're only seeing the protocol error in one them. We haven't seen the problem in any of the others, even though all applications a similar amount of load. The setup for our connections to Redis is in a shared codebase.

The only difference I can think of is that the application with the protocol error makes use of a pattern subscription. Something along the lines of this:

    var redis = require('redis');

    function connect(port, host) {
        var client = redis.createClient(port, host);
        client.psubscribe('PREFIX:*:*');
        client.on('pmessage', onMessageReceived);
        return client;
    }

I think existing subscriptions are reestablished after a disconnect. Maybe usable as starting point?

tal-beja commented 8 years ago

I have the same problem on node-redis-parser: v2.0.4

events.js:141
      throw er; // Unhandled 'error' event
      ^
ReplyError: Protocol error, got "1" as reply type byte. Please report this.
    at parseType (/var/app/current/node_modules/fw_pubsub/node_modules/redis/node_modules/redis-parser/lib/parser.js:226:34)

Always "1" as reply type byte by the way

BridgeAR commented 8 years ago

If you run into this with v.2.1.0 please provide all error properties. I extended the errors so it'll likely be possible to debug this afterwards.

BridgeAR commented 8 years ago

@tal-beja @arjenvanderende @tjsail33 did anyone of you run into the issue again?

zxfrank commented 8 years ago

I had this same issue and it appeared that there was a function that was re-broadcasting the message each time it was received, with 4 servers. It caused a memory increase then errors then crash. Since the code has been fixed, everything works well.

BridgeAR commented 7 years ago

As no one replied anymore I guess the issue was somewhere else / is already fixed.

If anyone runs into this again, please let me know and or reopen the issue.

manasovdan commented 7 years ago

Hey @BridgeAR I have ReplyError: Protocol error, got \"9\" as reply type byte. Please report this.\n at parseType (/opt/cloudcoreo/webapp/node_modules/redis-parser/lib/parser.js:293:17)

BridgeAR commented 7 years ago

@manasovdan please try to reproduce and provide all error information. I need at least the buffer and better also the offset attribute from the thrown error.

What operations do you use btw and do you know what commands were executed while running into that error?

manasovdan commented 7 years ago

@BridgeAR I'll try to reproduce that but the error is only on production. We use redis for caching and scaling websocket commands (simple PUB/SUB).

The full error message {"level":"error","message":"ReplyError: Protocol error, got \"9\" as reply type byte. Please report this.\n at parseType (/opt/cloudcoreo/webapp/node_modules/redis-parser/lib/parser.js:293:17)\n at JavascriptRedisParser.execute (/opt/cloudcoreo/webapp/node_modules/redis-parser/lib/parser.js:549:20)\n at Socket.<anonymous> (/opt/cloudcoreo/webapp/node_modules/redis/index.js:267:27)\n at emitOne (events.js:96:13)\n at Socket.emit (events.js:188:7)\n at readableAddChunk (_stream_readable.js:172:18)\n at Socket.Readable.push (_stream_readable.js:130:10)\n at TCP.onread (net.js:542:20)","timestamp":"2017-03-05T20:46:51.999Z"}

BridgeAR commented 7 years ago

@manasovdan I just published a new version that should help with debugging by more prominently show the buffer and offset data in the error. If you again run into the error, please provide that information :-)

I'm going to reopen the issue but it's really difficult to know what happened without any further information.

BridgeAR commented 7 years ago

As there's no new feedback, I'll close this again. If you have any further information, please let me know.