ArgumentError: string contains null byte

[pmontrasio]

I got this error attempting to send a message encoded with msgpack. Here are the details, from the rails console I was using. The zero bytes are introduced when msgpack encodes the number 1385134014291, which by the way is a timestamp with milliseconds.

$ bundle exec irb 
2.0.0p247 :001 > require "rubygems"
 => false 
2.0.0p247 :002 > require "rbczmq"
 => true 
2.0.0p247 :003 > require "msgpack"
 => true 
2.0.0p247 :004 > context = ZMQ::Context.new
 => #<ZMQ::Context:0x000000029cbb28> 
2.0.0p247 :005 > broker = context.socket :REQ
 => #<ZMQ::Socket::Req:0x000000029e6c48> 
2.0.0p247 :006 > broker.connect "tcp://localhost:5561"
 => true 
2.0.0p247 :007 > values = {"key1" => "value1", "key2" => "value2"}
 => {"key1"=>"value1", "key2"=>"value2"} 
2.0.0p247 :008 > msg = MessagePack.pack(["kkE3g5kCtLuer5q5ldvzD6TR3HYxia", 1385134014291, "2013-11-22T16:26:54+01:00", 2, "two", values])
 => "\x96\xBEkkE3g5kCtLuer5q5ldvzD6TR3HYxia\xCF\x00\x00\x01B\x80k\xBFS\xB92013-11-22T16:26:54+01:00\x02\xA3two\x82\xA4key1\xA6value1\xA4key2\xA6value2" 
2.0.0p247 :009 > broker.send msg
ArgumentError: string contains null byte
    from ***/.rvm/gems/ruby-2.0.0-p247@***/gems/rbczmq-1.7.1/lib/zmq/socket.rb:17:in `send'
    from ***/.rvm/gems/ruby-2.0.0-p247@***/gems/rbczmq-1.7.1/lib/zmq/socket.rb:17:in `send'
    from (irb):9
    from ***/.rvm/rubies/ruby-2.0.0-p247/bin/irb:13:in `<main>'

I'm using zeromq 3.2.4.

I've got two corresponding Python 2.7.3 and NodeJs 0.10.22 implementations that produce the same encoded message with the zero bytes and are successful to send it to the broker. Apparently they are able to cope with the zero bytes. That's unfortunate because I'm way more a Ruby developer than a Python or Node one.

if there is any need to check for interoperability, I built their environments with

pip install pyzmq
pip install msgpack-python

and

npm install zmq
npm install msgpack

respectively.

As a reference, somebody reported something similar on zmq and Ruby 1.8 four years ago http://lists.zeromq.org/pipermail/zeromq-dev/2009-July/001070.html

[methodmissing]

Thanks Paul - will look in a bit.

[mattconnolly]

Just found this old thread: https://www.ruby-forum.com/topic/1018901

In ZMQ::Socket#send we get the message to send as a C string which does not support null bytes... taking a quick look.

[mattconnolly]

In the above commit, I've stripped out the calls to CZMQ's sending string functions since that only supported null terminated C strings. Since ruby strings are also used for storing arbitrary binary data, I've made it send the raw bytes of the string.

Sound alright?

Note that there is no encoding translation. Strings are sent in their current encoding, and when received, they will be marked as ASCII_8BIT.

I've added a test for sending a string with a null byte in it too.

Not sure what's up with the rubinius bundler error in Travis though... :/

[methodmissing]

Awesome work Matt :-) I released 1.7.4 just now.

Closing the issue and will look into Rubinius @ Travis as well.

[pmontrasio]

I confirm that the new version works. Thank you!

[babaru]

However, the code below will trigger the same error.

require 'zmq'
require 'pp'

class DealerHandler < ZMQ::Handler
  def initialize(pollable, dealer)
    super
    @dealer = dealer
  end

  def on_readable
    @dealer.perform recv
  end
end

class RouterHandler < ZMQ::Handler
  def initialize(pollable, router)
    super
    @router = router
  end

  def on_writable
    @router.work
  end
end

class Dealer
  attr_reader :thread
  def initialize(ctx, endpoint, identity = "")
    @socket = ctx.socket(:DEALER)
    @socket.identity = identity
    @socket.connect(endpoint)
    # verbose output
    @socket.verbose = true
    @jobs, @working = 0, 0.0
  end

  def start
    ZL.register_readable(@socket, DealerHandler, self)
    self
  end

  def stop
    ZL.remove(@socket)
    stats
  end

  def perform(work)
    # Random hot loop to simulate CPU intensive work
    start = Time.now
    work.to_i.times{}
    @jobs += 1
    @working += (Time.now - start).to_f
  end

  private
  def stats
    puts "Processed #{@jobs} jobs in %.4f seconds" % @working
    $stdout.flush
  end
end

class Router
  def initialize(ctx, endpoint, identity = "")
    @ctx, @endpoint, @identity, @dealers = ctx, endpoint, identity, []
    @socket = ctx.socket(:ROUTER)
    # verbose output
    @socket.verbose = true
    @socket.bind(endpoint)
    @socket.linger = 1
    @interrupted = false
  end

  def spawn_dealers(count = 10)
    count.times do
      @dealers << Dealer.new(@ctx, @endpoint, @identity).start
    end
  end

  def start
    ZL.register_writable(@socket, RouterHandler, self)
  end

  def stop
    @dealers.each{|c| c.stop }
    ZL.remove(@socket)
    ZL.stop
    @ctx.destroy
  end

  def work
    work = [1,0,1,2,3,4,5].pack('c*')
    @socket.sendm(@identity)
    @socket.send(work)
  end
end

ZL.run do
  ctx = ZMQ::Context.new
  router = Router.new(ctx, "inproc://routing-flow.test", 'xyz')
  router.spawn_dealers
  trap(:INT) do
    router.stop
  end
  router.start
end

Please find details output as follows:

15-07-16 23:54:41 I: ROUTER socket 0x7fd29b9d59c0: bound "inproc://routing-flow.test"
15-07-16 23:54:41 I: ROUTER socket 0x7fd29b9d59c0: set option "LINGER" 1
15-07-16 23:54:41 I: ROUTER socket 0x7fd29b8a1e00: sendm "xyz"
router.rb:94:in `send': string contains null byte (ArgumentError)
    from router.rb:94:in `work'
    from router.rb:22:in `on_writable'
    from /Users/apple/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/rbczmq-1.7.9/lib/zmq/loop.rb:33:in `start'
    from /Users/apple/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/rbczmq-1.7.9/lib/zmq/loop.rb:33:in `run'
    from router.rb:98:in `<main>'

It looks like in the class PollItem the implementation is somehow different? Could you please have a look at this issue? Thank you.

[babaru]

I'm using version 1.7.9 of rbczmq, btw.

[babaru]

@methodmissing could you please look into this? Thanks a lot.