Coredump after to_binary() => to_term() roundtrip

[evnu]

I am able to reproduce a core dump with the simple roundtrip() function below (see https://github.com/evnu/rustler_core_dump). The function takes a term, encodes it into a binary, reencodes into a term and returns that term wrapped with a 1 within a tuple.

#[macro_use]
extern crate rustler;

use rustler::{NifEncoder, NifEnv, NifResult, NifTerm};

rustler_export_nifs! {
    "Elixir.RustlerCoreDump",
    [
        ("roundtrip", 1, roundtrip),
    ],
    None
}

fn roundtrip<'a>(env: NifEnv<'a>, args: &[NifTerm<'a>]) -> NifResult<NifTerm<'a>> {
    let original: NifTerm = args[0].decode()?;
    let binary = original.to_binary();
    let roundtripped: NifTerm = binary.to_term(env);
    Ok((1, roundtripped).encode(env))
}

When I use the following Elixir implementation to load the nif, running test results in a core dump.

defmodule RustlerCoreDump do
  use Rustler, otp_app: :rustler_core_dump, crate: "rustler_core_dump"

  def roundtrip(term), do: throw(:nif_not_loaded)

  def test do
    reference = make_ref()
    IO.inspect(reference)
    {1, reference} = roundtrip(reference)
    IO.inspect(reference)
  end
end

Note that inspecting the resulting reference assigned after roundtrip seems to be crucial: I need to either IO.inspect(reference) or match with {1, ^reference} = roundtrip(reference) to reproduce this.

Running the Example

 mix run -e RustlerCoreDump.test           
==> rustler                             
Compiling 1 file (.yrl)
Compiling 1 file (.xrl)
Compiling 2 files (.erl)
Compiling 6 files (.ex)
Generated rustler app
==> rustler_core_dump
Compiling NIF crate :rustler_core_dump (native/rustler_core_dump)...
    Finished release [optimized] target(s) in 0.0 secs
Compiling 1 file (.ex)
Generated rustler_core_dump app
#Reference<0.3640706538.2123366403.78090>
[1]    7004 segmentation fault (core dumped)  mix run -e RustlerCoreDump.test

Version Information

rust:

rustc -V
rustc 1.26.0 (a77568041 2018-05-07)

Elixir:

elixirc -v
Erlang/OTP 20 [erts-9.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]

Elixir 1.6.4 (compiled with OTP 20)

mix.lock:

%{
  "rustler": {:git, "https://github.com/hansihe/rustler", "00bcc871cdacc70af35ed29daeb9e3f37cd3a1f4", [sparse: "rustler_mix"]},
}

[evnu]

I added some debugging output:

fn roundtrip<'a>(env: NifEnv<'a>, args: &[NifTerm<'a>]) -> NifResult<NifTerm<'a>> {
    let original: NifTerm = args[0].decode()?;
    eprintln!("{:?}", original);
    let binary = original.to_binary();
    let roundtripped: NifTerm = binary.to_term(env);
    eprintln!("{:?}", roundtripped);
    Ok((1, roundtripped).encode(env))
}

Resulting run:

mix run -e RustlerCoreDump.test                                
Compiling NIF crate :rustler_core_dump (native/rustler_core_dump)...
   Compiling rustler_core_dump v0.1.0 (file:///home/mo/tools/rustler_core_dump/native/rustler_core_dump)
    Finished dev [unoptimized + debuginfo] target(s) in 0.47 secs
#Reference<0.441875240.1587806209.52051>
#Ref<0.441875240.1587806209.52051>
<cp/header:0x0000000000000000>

[evnu]

A similar bug was fixed with OTP 20.3.7:

OTP-15080 Application(s): erts

          Fixed bug in enif_binary_to_term which could cause
          memory corruption for immediate terms (atoms, small
          integers, pids, ports, empty lists).

I can still trigger the segfault with the example above.

[hansihe]

@evnu Could you test it again on latest master?

[scrogson]

ah, you beat me to it @hansihe

[evnu]

The example does not segfault any more, thank you! I needed to adapt the conversion into a Term as to_binary() now returns an OwnedBinary:

    let roundtripped: Term = binary.release(env).to_term(env);

Note that the resulting roundtripped does not equal the value put into the NIF:

    eprintln!("{:?}", original == roundtripped); #=> false

Converting it within Elixir with :erlang.binary_to_term/1 produces the original value again, though. I guess I would need to use env::binary_to_term() to convert back into the original term.