Backtrace and evaluation doesn't work because of null bytes

[x-yuri]

Problem description

Backtrace and evaluation doesn't always work, supposedly because of null bytes.

Expected behavior

Null bytes are not in byebug's way.

Actual behavior

See below.

Steps to reproduce the problem

1.sh:

#!/bin/sh
set -eux
apk add build-base
gem install byebug
echo '
    source "https://rubygems.org"
    gem "tzinfo", "1.2.7"
    gem "thread_safe", "0.0.3"
' > Gemfile
sed -Ei '
    /def initialize.*/ a\
        require "byebug"; debugger
' /usr/local/lib/ruby/2.6.0/bundler/vendor/molinillo/lib/molinillo/errors.rb
echo 'where
conflicts' | bundle

$ docker run --rm -itv $PWD:/app -w /app ruby:2.6-alpine3.11 ./1.sh
...
+ echo 'where
conflicts'
+ bundle
Fetching gem metadata from https://rubygems.org/...
Resolving dependencies...
[65, 74] in /usr/local/lib/ruby/2.6.0/bundler/vendor/molinillo/lib/molinillo/errors.rb
   65:     # Initializes a new error with the given version conflicts.
   66:     # @param [{String => Resolution::Conflict}] conflicts see {#conflicts}
   67:     # @param [SpecificationProvider] specification_provider see {#specification_provider}
   68:     def initialize(conflicts, specification_provider)
   69:         require "byebug"; debugger
=> 70:       pairs = []
   71:       Compatibility.flat_map(conflicts.values.flatten, &:requirements).each do |conflicting|
   72:         conflicting.each do |source, conflict_requirements|
   73:           conflict_requirements.each do |c|
   74:             pairs << [c, source]
(byebug) where
*** string contains null byte
(byebug) conflicts
*Error in evaluation*

In both cases the following error gets triggered:

ArgumentError: string contains null byte

And that happens when byebug tries to inspect the conflicts variable. The variable goes along the lines of:

{"thread_safe" => #<struct Bundler::Molinillo::Resolver::Resolution::Conflict
    ...
    activated_by_name = {
        "ruby\u0000" => #<Bundler::Resolver::SpecGroup:0x00007fdc2b0a8ae8
            @name = "ruby\u0000",
            @version = #<Gem::Version "2.6.6.146">,
            @specs = {"ruby" => #<Gem::Specification:0x00007fdc2b1084e8
                @name = "ruby\u0000",
                @full_name = "ruby\u0000-2.6.6.146",
                ...},
            ...>,
        "rubygems\u0000" => #<Bundler::Resolver::SpecGroup:0x00007fdc2b0a8340
            @name = "rubygems\u0000",
            @version = #<Gem::Version "3.0.3">,
            @specs = {"ruby" => #<Gem::Specification:0x00007fdc2b103da8
                @name = "rubygems\u0000",
                @full_name = "rubygems\u0000-3.0.3",
                ...},
            ...>,
        ...},
    ...}

---

UPD

Without bundler the ArgumentError (string contains null byte) can be reproduced this way:

1.sh:

#!/bin/sh
set -eux
apk add build-base
gem install byebug
echo '
    class A
      def initialize(specs)
        @specs = specs
      end
    end

    class B
      def initialize(name)
        @name = name
      end

      def inspect
        "#{super[0..-2]} #{@name}>"
      end
    end

    p A.new(B.new("ruby\u0000"))
' > 1.rb
ruby 1.rb

$ docker run --rm -v $PWD:/app -w /app ruby:2.6-alpine3.11 ./1.sh
...
+ ruby 1.rb
1.rb:18:in `inspect': string contains null byte (ArgumentError)
        from 1.rb:18:in `p'
        from 1.rb:18:in `<main>'

So, supposedly ruby expects inspect to not produce strings with null bytes. But rubygems does. Should I file a rubygems issue? ruby(gems)?\0 seem to first appear here for no apparent reason.

But null checks are not always performed. They are performed e.g. when inspecting an instance variable:

inspect_i rb_str_catf rb_str_vcatf BSD_vfprintf ruby__sfvextra rb_string_value_cstr

but not an object itself, or a hash of objects:

p A.new(B.new("ruby\u0000"))        # fails
p B.new("ruby\u0000")               # succeeds, but contains \0
p {"ruby" => B.new("ruby\u0000")}   # succeeds, but contains \0

Evaluation can be worked around like so:

a = A.new(B.new("ruby\u0000"))

def my_inspect v
  begin
    v.inspect
  rescue ArgumentError
    c = v.class
    addr = v.object_id << 1
    ivars = v.instance_variables.map do |k|
      " #{k}=#{my_inspect(v.instance_variable_get(k))}"
    end
    sprintf "<#%s:0x%x%s>", c, addr, ivars.join('')
  end
end

puts my_inspect a   # <#A:0x7f23be286fe8 @specs=#<B:0x00007f23be287010 @name="ruby\u0000" ruby>>

But at least the object address is not padded with 0's, and possibly other issues.

Another way to reproduce the backtrace issue:

2.sh:

#!/bin/sh
set -eux
apk add build-base
gem install byebug
echo '
    require "byebug"

    class A
      def initialize(specs)
        @specs = specs
      end
    end

    class B
      def initialize(name)
        @name = name
      end

      def inspect
        "#{super[0..-2]} #{@name}>"
      end
    end

    def f a
        a.tap { |v|
          debugger
        }
    end

    f [A.new(B.new("ruby\u0000"))]  # or hash, but not just the object

' > 2.rb
echo w | ruby 2.rb

$ docker run --rm -itv $PWD:/app -w /app ruby:2.6-alpine3.11 ./2.sh
...
+ echo w
+ ruby 2.rb
Return value is: nil

[18, 27] in /app/2.rb
   18:     end
   19:
   20:     def f a
   21:         a.tap { |v|
   22:           debugger
=> 23:         }
   24:     end
   25:
   26:     f [A.new(B.new("ruby\u0000"))]
   27:
(byebug) w
*** string contains null byte

The error gets triggered here, when trying to do a.to_s. That wouldn't happen w/o tap or something, because the frame has to have no binding for that line to be evaluated (whatever that means).