uncaught error: 348 trailing bytes

[zacharynevin]

I am using socket.io-emitter to broadcast an event to a set of channels with a for loop:

In the file, I have:

// in the file
var io = require('socket.io-emitter')({
  host: 'localhost',
  port: 6379
});

module.exports = {
    exampleFunction: function(req, res, next) {
      var channels = req.param('channels'),
            data = req.param('data');

      for (var i=0; i<channels.length; i++) {
        io.to(channels[i]).emit('example event', data)
      }
    }

}

In app.js, I have socket.io-redis:

io.adapter(socketio_redis({ 
  host: 'localhost', 
  port: 6379,
  pubClient: redis.createClient(6379, '127.0.0.1'),
  subClient: redis.createClient(6379, '127.0.0.1')
}))

When I try to run exampleFunction, I get the following uncaught error in my console:

Error: 348 trailing bytes
    at Object.decode (C:\Users\Zachary\Documents\GitHub\thegraduate_backend\node
_modules\socket.io-redis\node_modules\msgpack-js\msgpack.js:200:47)
    at Redis.onmessage (C:\Users\Zachary\Documents\GitHub\thegraduate_backend\no
de_modules\socket.io-redis\index.js:93:24)
    at RedisClient.EventEmitter.emit (events.js:106:17)
    at RedisClient.return_reply (C:\Users\Zachary\Documents\GitHub\thegraduate_b
ackend\node_modules\redis\index.js:672:22)
    at ReplyParser.<anonymous> (C:\Users\Zachary\Documents\GitHub\thegraduate_ba
ckend\node_modules\redis\index.js:309:14)
    at ReplyParser.EventEmitter.emit (events.js:95:17)
    at ReplyParser.send_reply (C:\Users\Zachary\Documents\GitHub\thegraduate_bac
kend\node_modules\redis\lib\parser\javascript.js:300:10)
    at ReplyParser.execute (C:\Users\Zachary\Documents\GitHub\thegraduate_backen
d\node_modules\redis\lib\parser\javascript.js:211:22)
    at RedisClient.on_data (C:\Users\Zachary\Documents\GitHub\thegraduate_backen
d\node_modules\redis\index.js:534:27)
    at Socket.<anonymous> (C:\Users\Zachary\Documents\GitHub\thegraduate_backend
\node_modules\redis\index.js:91:14)

When I change data to be something smaller, like:

var data = { testing: true }

I get the same error, but to a smaller number of trailing bytes. e.g. 95 trailing bytes.

[alexprice1]

So we have narrowed this down. https://github.com/automattic/socket.io-emitter#emitterclient-opts says to set return_buffers to true on your redis client. This should fix this issue.

We need to add support to socket.io-redis to see if the incoming/outcoming data is binary before sending it to msgpack.

[zacharynevin]

It worked! Thank you!

[hems]

maybe on the README we could "enforce" a bit more that parameter!

i also missed the detect_buffers: true on the "Custom Client" section

[nnmduc]

I have same problem. I use ruby socket emitter to broadcast an event:

emitter = SocketIO::Emitter.new(key: 'socket.io.dev', redis: Redis.new(url: 'redis://localhost:6379'))
data = {id: 100, username: 'peternguyen', content: Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s'}
emitter.emit('comment', data)

In app.js

var io = require('socket.io')(5001);
var request = require('request');
var redis = require('redis');
var redis_adapter = require('socket.io-redis');

var pub = redis.createClient(6379, 'localhost', {return_buffers: true, auth_pass: ''});
var sub = redis.createClient(6379, 'localhost', {return_buffers: true, auth_pass: ''});

io.adapter(redis_adapter({
    key: 'socket.io.dev',
    pubClient: pub,
    subClient: sub
}));

I get the following in my console:

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: -2340102607 trailing bytes
    at Object.decode (/home/peter/node_modules/socket.io-redis/node_modules/msgpack-js/msgpack.js:200:47)
    at Redis.onmessage (/home/peter/node_modules/socket.io-redis/index.js:94:24)
    at RedisClient.emit (events.js:106:17)
    at RedisClient.return_reply (/home/peter/node_modules/redis/index.js:696:22)
    at ReplyParser.<anonymous> (/home/peter/node_modules/redis/index.js:321:14)
    at ReplyParser.emit (events.js:95:17)
    at ReplyParser.send_reply (/home/peter/node_modules/redis/lib/parser/javascript.js:300:10)
    at ReplyParser.execute (/home/peter/node_modules/redis/lib/parser/javascript.js:211:22)
    at RedisClient.on_data (/home/peter/node_modules/redis/index.js:547:27)
    at Socket.<anonymous> (/home/peter/node_modules/redis/index.js:102:14)

[hems]

peter chexk the return_buffers option, that should sort your issue.

[Thesephi]

@peternguyen1991 please refer here: https://github.com/Automattic/socket.io-redis/pull/61#commits-pushed-8d7b40f

Your problem was that Ruby's SocketIO::Emitter packs the messages using the new msgpack-js v5, while the official socket.io-redis client only supports traditional msgpack-js. You could either clone from @burnssun's brilliant fork or manually edit your local socket.io-redis node module to reflect the necessary changes.

[aschmid]

i saw the trailing bytes issue the first time on my app today. when i set the return buffers option i get this error

TypeError: Object socket.io#VZCddR has no method 'split'

  at Redis.onmessage (/path/to/my/app/node_modules/socket.io-redis/index.js:92:26)

is there a permanent fix available for all these issues?

[aschmid]

ok found the issue. node redis v2.0.0 (released yesterday) has issues with the latest version of socket.io-redis causing the trailing bytes exeption. when setting return_buffers to true unfortunately it throws an exception like in my previous comment.

[stephengardner]

@aschmid so is the solution to simply rollback one of these modules versions, or did you find another way?

[aschmid]

@stephengardner i didn't have time to look into a 'proper' solution yet so my fix was to use v1.0.0 of https://www.npmjs.com/package/redis.

[PerMoeller]

In my setup, node 4.1.1, socket-io 0.1.4, node_redis 2.0.1 the problem is that index.js line 92 expect "channel" to be a string, but it's a Buffer.

[Mokbii]

set up redis ver. 1.0.0... is trailing bytes error fixed..

i think, that issue is node_redis 2.0.1 bug.

modify your file(pakage.json) redis version " 1.* " in dependencies

[zacharynevin]

I am still getting this issue even when I set my redis version to 1.0.0

[zacharynevin]

I used the same package versions as PerMoeller and got his exact issue:

channel.split("#") <!-- cannot read split of undefined

[vodolaz095]

I'm still having this error with redis client of 2.2.3 and socket.io of 1.3.7, socket.io-redis:0.1.4, even with the detect_buffers and return_buffers set to true https://github.com/vodolaz095/hunt/blob/master/lib/datastore/redisClient.js#L44 .

/home/projects/vodolaz095/hunt/node_modules/socket.io-redis/node_modules/msgpack-js/msgpack.js:200
  if (decoder.offset !== buffer.length) throw new Error((buffer.length - decoder.offset) + " trailing bytes");
                                        ^

Error: 75 trailing bytes
    at Object.decode (/home/projects/vodolaz095/hunt/node_modules/socket.io-redis/node_modules/msgpack-js/msgpack.js:200:47)
    at Redis.onmessage (/home/projects/vodolaz095/hunt/node_modules/socket.io-redis/index.js:94:24)
    at emitThree (events.js:97:13)
    at RedisClient.emit (events.js:175:7)
    at RedisClient.return_reply (/home/projects/vodolaz095/hunt/node_modules/redis/index.js:622:22)
    at /home/projects/vodolaz095/hunt/node_modules/redis/index.js:310:18
    at doNTCallback0 (node.js:417:9)
    at process._tickDomainCallback (node.js:387:13)

npm ERR! Linux 4.1.8-100.fc21.x86_64
npm ERR! argv "/usr/bin/node" "/bin/npm" "start"
npm ERR! node v4.2.1
npm ERR! npm  v2.14.7
npm ERR! code ELIFECYCLE
npm ERR! hunt@0.6.0 start: `node examples/index.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the hunt@0.6.0 start script 'node examples/index.js'.
npm ERR! This is most likely a problem with the hunt package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node examples/index.js
npm ERR! You can get their info via:
npm ERR!     npm owner ls hunt
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /home/projects/vodolaz095/hunt/npm-debug.log

And it works ok with redis of 1.0.0.

[Caligone]

Same issue here with redis v2.2.5, socket.io v1.3.7 and socket.io-redis v0.1.4.

// Server
redis.createClient(redisOptions.port, redisOptions.host, {
      detect_buffers: true,
      return_buffers: true,
      auth_pass: redisOptions.password
    });
// Client
socket.emit('echo', {content: 'A'});

And I'm getting this error:

/app/node_modules/socket.io-redis/node_modules/msgpack-js/msgpack.js:200
  if (decoder.offset !== buffer.length) throw new Error((buffer.length - decoder.offset) + " trailing bytes");
                                        ^

Error: 86 trailing bytes
    at Object.decode (/app/node_modules/socket.io-redis/node_modules/msgpack-js/msgpack.js:200:47)
    at Redis.onmessage (/app/node_modules/socket.io-redis/index.js:94:24)
    at emitThree (events.js:97:13)
    at RedisClient.emit (events.js:175:7)
    at RedisClient.return_reply (/app/node_modules/redis/index.js:620:22)
    at /app/node_modules/redis/index.js:309:18
    at doNTCallback0 (node.js:417:9)
    at process._tickCallback (node.js:346:13)

Hope it helps

[austinh]

I have this issue with the following versions

node: v4.2.1 socket.io: 1.37 socket.io-redis: 0.14 redis: 1.0 (also tried 2.0 and 2.3)

I am emitting an event with the socket.io-php-emitter with msgpack and redis

events.js:141
      throw er; // Unhandled 'error' event
      ^

Error: -842817745 trailing bytes
    at Object.decode (/var/www/website1/lib/vendor/node/node_modules/socket.io-redis/node_modules/msgpack-js/msgpack.js:200:47)
    at Redis.onmessage (/var/www/website1/lib/vendor/node/node_modules/socket.io-redis/index.js:94:24)
    at emitThree (events.js:97:13)
    at RedisClient.emit (events.js:175:7)
    at RedisClient.return_reply (/var/www/website1/lib/vendor/node/node_modules/socket.io-redis/node_modules/redis/index.js:672:22)
    at HiredisReplyParser.<anonymous> (/var/www/website1/lib/vendor/node/node_modules/socket.io-redis/node_modules/redis/index.js:309:14)
    at emitOne (events.js:77:13)
    at HiredisReplyParser.emit (events.js:169:7)
    at HiredisReplyParser.execute (/var/www/website1/lib/vendor/node/node_modules/socket.io-redis/node_modules/redis/lib/parser/hiredis.js:43:18)
    at RedisClient.on_data (/var/www/website1/lib/vendor/node/node_modules/socket.io-redis/node_modules/redis/index.js:534:27)

[austinh]

Update: Switching dependenices from msgpack-js to msgpack-js-v5 fixed this problem for me. It looks like php's msgpack module has been updated to the spec, but the version used by msgpack has not.

[laukaichung]

Does this bug have anything to do with redis version after 3.0.6? I have tried the suggestions in this post and downgraded all modules but the problem isn't fixed

[StevenRoan]

After several experiments, my service works fine with

    "socket.io": "1.3.7",
    "socket.io-redis": "0.1.4",

Just for reference

[terion-name]

Same error here. Nothing fixes :( Nor return_buffers, nor version suggested by @StevenRoan

[xyzlast]

I've same problem.

but the version suggested by @StevenRoan is works fine.

I've test it using socket.io-reids version is 1.0.0.

socket.io: 1.4.7
socket.io-redis: 1.0.0

case 1. detect_buffer: true, return_buffers: true

NoError. But Subscription is not working.

case 2. detect_buffer: true, return_buffers: false

message Pack Error.

if (decoder.offset !== buffer.length) throw new Error((buffer.length - decoder.offset) + " trailing bytes");

case 3. detect_buffer: false, return_buffers: true

Same case 2.

case 3. detect_buffer: false, return_buffers: false

Same case 2.

[xyzlast]

Resolved it. It's my mistake.;;

detect_buffer, return_buffer options are not same in Pub and Sub. @StevenRoan, @terion-name check your options in Pub/Sub Clients.

here is my code that is working.

    "socket.io": "^1.4.5",
    "socket.io-redis": "^1.0.0",

function create(options) {
  if (config.redis.password) {
    options.auth_pass = config.redis.password;
  }
  const client = require('redis').createClient(config.redis.port, config.redis.host, options);
  const promise = require('bluebird');
  promise.promisifyAll(client);
  return client;
}

function createClient() {
  const options = {
    detect_buffers: true,
    return_buffers: false
  };
  return create(options);
}

function createPub() {
  const options = {
    detect_buffers: true,
    return_buffers: false
  };
  return create(options);
}

function createSub() {
  const options = {
    return_buffers: true
  };
  return create(options);
}

[FabioAntunes]

I'm also having the same issue, and it's very easy to replicate.

index.js:

var io = require('socket.io')(3000);
var redisIO = require('socket.io-redis');
var redis = require('redis');
var config = {
  redis: {
    host: '10.0.1.231',
    port: 6379
  },
  room: 'client1'
};

var pub = redis.createClient(config.redis.port, config.redis.host, {return_buffers: true});
var sub = redis.createClient(config.redis.port, config.redis.host, {return_buffers: true});

io.adapter(redisIO({
  host: config.redis.host,
  port: config.redis.port,
  pubClient: pub,
  subClient: sub
}));

io.on('connection', function(socket) {
  console.log('connected');
  socket.on('getMessages', function(){
    console.log('published');
    pub.publish('coolMessages', 'yolo message');
  });
});

sub.subscribe('coolMessages');

sub.on('message', function(channel, data){
  console.log('message!'); // this doesn't run
});

package.json:

{
  "name": "test_dm",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "MIT",
  "dependencies": {
    "redis": "^2.3.0",
    "socket.io": "^1.4.5",
    "socket.io-redis": "^1.0.0"
  }
}

just creat these two files, do a npm install.

Then for the client: html

<!doctype html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Socket.IO Chat Example</title>
  <link rel="stylesheet" href="style.css">
</head>
<body>
  <ul class="pages">
    <li class="chat page">
      <div class="chatArea">
        <ul class="messages"></ul>
      </div>
      <input class="inputMessage" placeholder="Type here..."/>
    </li>
    <li class="login page">
      <div class="form">
        <h3 class="title">What's your nickname?</h3>
        <input class="usernameInput" type="text" maxlength="14" />
      </div>
    </li>
  </ul>

  <script src="https://code.jquery.com/jquery-1.10.2.min.js"></script>
  <script src="bower_components/socket.io-client/socket.io.js"></script>
  <script>
    $(function() {
      var socket = io('http://localhost:3000');

      socket.on('connect', function () {
        console.log(socket.id);
      });

      socket.emit('getMessages')
    });
  </script>
</body>
</html>

bower.json

{
  "name": "socket-test",
  "description": "",
  "main": "",
  "moduleType": [],
  "license": "MIT",
  "homepage": "",
  "ignore": [
    "**/.*",
    "node_modules",
    "bower_components",
    "test",
    "tests"
  ],
  "dependencies": {
    "socket.io-client": "^1.4.5"
  }
}

Just create also these two files and to a bower install.

[agix]

Hi ! Actually the bug exists if you use the same redis client in socket.io-redis and in a "classic" usage. Just create two more redisClient to pub and sub.

var ioPub = redis.createClient(redisPort, redisHost, {detect_buffers: true, return_buffers: false});
var ioSub = redis.createClient(redisPort, redisHost, {return_buffers: true});

var pub = redis.createClient(redisPort, redisHost, {detect_buffers: true, return_buffers: false});
var sub = redis.createClient(redisPort, redisHost, {detect_buffers: true, return_buffers: false});

io.adapter(redisIO({
  host: config.redis.host,
  port: config.redis.port,
  pubClient: ioPub,
  subClient: ioSub
}));

io.on('connection', function(socket) {
  console.log('connected');
  socket.on('getMessages', function(){
    console.log('published');
    pub.publish('coolMessages', 'yolo message');
  });
});

sub.subscribe('coolMessages');

sub.on('message', function(channel, data){
  console.log('message!'); // this doesn't run
});

[Audio]

I had this issue with redis@2.4.2, but it seem to be okay after upgrading to v2.5.0

https://github.com/NodeRedis/node_redis/releases

[ucola]

I have the same error with version redis@2.6.0-2 socket.io@1.4.6

/home/*/npm_modules/node_modules/socket.io-redis/node_modules/msgpack-js/msgpack.js:200 if (decoder.offset !== buffer.length) throw new Error((buffer.length - decoder.offset) + " trailing bytes");

var redis = require('/home/' + _servername + '/npm_modules/node_modules/redis').createClient; var adapter = require('/home/' + _servername + '/npm_modules/node_modules/socket.io-redis'); var pub = redis(_redisport, _redishost, { auth_pass: _redispass }); var sub = redis(_redisport, _redishost, { detect_buffers: true, auth_pass: _redispass }); io.adapter(adapter({ pubClient: pub, subClient: sub }));

did anybody have a solution for this issue? with redis@1.0.0 and socket.io@1.3.4 it works fine

[austinkelleher]

+1 I am experiencing this issue as well

[garfieldhamilton]

I was able to install the v5 version of msgpack-js-v5, npm i msgpack-js-v5 --save

then change the requires statement in node_modules/socket.io-redis/index.js (around line 8) from var msgpack = require('msgpack-js'); to var msgpack = require('msgpack-js-v5');

and that worked in my setup.

[ucola]

@garfieldhamilton can you tel us the version of redis and socket.io that you use?

[CostaRico]

+1

[mjasnikovs]

@garfieldhamilton msgpack-js-v5 is not passing tests, use msgpack5

[blalop]

For those still experimenting this issue (I truly hope no one is still using these versions) the following versions made the fix for me:

        "redis": "1.0.0",
        "socket.io": "3.0.0",
        "socket.io-redis": "4.0.1",

No additional configuration required.