danielpclark / rutie

“The Tie Between Ruby and Rust.”
MIT License
942 stars 62 forks source link

Thread::call_with_gvl causes pointer being freed was not allocated #69

Closed dsander closed 5 years ago

dsander commented 5 years ago

Hi,

I am using rutie to create a ruby gem for a rust library that basically fetches units of work from a server and then calls ruby classes to process those units. To make concurrent I am using Thread::call_without_gvl to release the GVL while doing network operations in rust and then Thread::call_with_gvl to make ruby calls. I could probably restructure the code to avoid the call_with_gvl call, but the 'static requirement for the closures makes it a bit tricky.

When calling the method below from ruby it fails with

ruby(9818,0x117bcb5c0) malloc: *** error for object 0x7fa850ef40a0: pointer being freed was not allocated
ruby(9818,0x117bcb5c0) malloc: *** set a breakpoint in malloc_error_break to debug

The weird thing is that the rust test works without issues.

Rust source ```rust #[macro_use] extern crate rutie; use rutie::{Class, Object, RString, Thread, VM}; class!(RutieExample); methods!( RutieExample, _itself, fn work(_input: RString) -> RString { let worker_class = "Object".to_string(); let b = Thread::call_without_gvl( move || { // Releasing the GVL here to do network IO // After we fetched a unit of work I wanto to call a ruby object Thread::call_with_gvl(move || { // Uncommenting this line "fixes" the issue // let worker_class = "Object".to_string(); let ruby_worker = Class::from_existing(&worker_class); ruby_worker.send("name", None) }) }, Some(|| {}), ); match b.try_convert_to::() { Ok(ruby_string) => ruby_string, Err(_) => RString::new_utf8("Fail!"), } } ); #[allow(non_snake_case)] #[no_mangle] pub extern "C" fn Init_rutie_ruby_example() { Class::new("RutieExample", None).define(|itself| { itself.def_self("work", work); }); } #[test] fn it_works() { // Rust projects must start the Ruby VM VM::init(); Init_rutie_ruby_example(); let ruby_worker = Class::from_existing("RutieExample"); let res = ruby_worker.send("work", None); let res = match res.try_convert_to::() { Ok(ruby_string) => ruby_string, Err(_) => RString::new_utf8("Fail!"), }; assert_eq!("Object", &res.to_string()); } ```
lldb stack strace ```(lldb) target create ~/.rbenv/versions/2.5.3/bin/ruby Current executable set to '~/.rbenv/versions/2.5.3/bin/ruby' (x86_64). (lldb) breakpoint set --name malloc_error_break Breakpoint 1: where = libsystem_malloc.dylib`malloc_error_break, address = 0x00000000000170de (lldb) run -Ilib fail.rb Process 8370 launched: '/Users/dominik/.rbenv/versions/2.5.3/bin/ruby' (x86_64) ruby(8370,0x1000b95c0) malloc: *** error for object 0x10070d790: pointer being freed was not allocated ruby(8370,0x1000b95c0) malloc: *** set a breakpoint in malloc_error_break to debug Process 8370 stopped * thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1 frame #0: 0x00007fff5e9a00de libsystem_malloc.dylib`malloc_error_break libsystem_malloc.dylib`malloc_error_break: -> 0x7fff5e9a00de <+0>: pushq %rbp 0x7fff5e9a00df <+1>: movq %rsp, %rbp 0x7fff5e9a00e2 <+4>: nop 0x7fff5e9a00e3 <+5>: nopl (%rax) Target 0: (ruby) stopped. (lldb) up frame #1: 0x00007fff5e993676 libsystem_malloc.dylib`malloc_vreport + 437 libsystem_malloc.dylib`malloc_vreport: 0x7fff5e993676 <+437>: cmpb $0x1, 0x32c47d6f(%rip) ; debug_mode + 3 0x7fff5e99367d <+444>: jne 0x7fff5e9936a3 ; <+482> 0x7fff5e99367f <+446>: leaq 0x18458(%rip), %rsi ; "*** sending SIGSTOP to help debug\n" 0x7fff5e993686 <+453>: movl $0x5, %edi (lldb) up frame #2: 0x00007fff5e9934a3 libsystem_malloc.dylib`malloc_report + 152 libsystem_malloc.dylib`malloc_report: 0x7fff5e9934a3 <+152>: movq 0x32c45b66(%rip), %rax ; (void *)0x00007fff915c6070: __stack_chk_guard 0x7fff5e9934aa <+159>: movq (%rax), %rax 0x7fff5e9934ad <+162>: cmpq -0x8(%rbp), %rax 0x7fff5e9934b1 <+166>: jne 0x7fff5e9934bc ; <+177> (lldb) up frame #3: 0x000000010194709a librutie_ruby_example.dylib`alloc::alloc::dealloc::hef07b11e979696ba(ptr="", layout=Layout @ 0x00007ffeefbfeb40) at alloc.rs:96 (lldb) up frame #4: 0x00000001019470f9 librutie_ruby_example.dylib`alloc::alloc::box_free::h1a8734277b8c0de7(ptr=Unique @ 0x00007ffeefbfeb60) at alloc.rs:206 (lldb) up frame #5: 0x00000001019464ad librutie_ruby_example.dylib`rutie::util::ptr_to_data::h4762f8239518f213(ptr=0x000000010070d790) at util.rs:101 98 99 pub unsafe fn ptr_to_data(ptr: *mut c_void) -> R { 100 *Box::from_raw(ptr as *mut R) -> 101 } (lldb) up frame #6: 0x000000010194715c librutie_ruby_example.dylib`rutie::binding::thread::call_with_gvl::hbebc4468eca2e309(func=) at thread.rs:92 89 util::closure_to_ptr(func), 90 ); 91 -> 92 util::ptr_to_data(ptr) 93 } 94 } 95 (lldb) up frame #7: 0x0000000101946c08 librutie_ruby_example.dylib`rutie::class::thread::Thread::call_with_gvl::h24af48b7283f4583(func=) at thread.rs:138 135 where 136 F: 'static + FnOnce() -> R, 137 { -> 138 thread::call_with_gvl(func) 139 } 140 } 141 (lldb) up frame #8: 0x0000000101946458 librutie_ruby_example.dylib`rutie_ruby_example::work::_$u7b$$u7b$closure$u7d$$u7d$::h3a14ca771a91723e at lib.rs:17 14 move || { 15 // Releasing the GVL here to do network IO 16 // After we fetched a unit of work I wanto to call a ruby object -> 17 Thread::call_with_gvl(move || { 18 // Uncommenting this line "fixes" the issue 19 // let worker_class = "Object".to_string(); 20 let ruby_worker = Class::from_existing(&worker_class); (lldb) ```
danielpclark commented 5 years ago

Thank you for submitting this issue. I will research this and see what I can learn about it. It will take some time for me to give you an answer or an update to help you.

Right now I'm looking in to the topic of an item not living long enough as I vaguely remember something written about it. I think, if I remember correctly, to help the lifetime of the item live longer you could do something like:

let _ = worker_class;

At the end of the code block where you need it to live longer. This will give a hint to the compiler about the scope of the lifetime that that object needs to live. I think this is your best bet to fix this.

If that's not the answer that's okay, I'm still looking in to it.

Also if you want to completely transfer ownership of a pointer over to Ruby you can use Box::into_raw to disable Rust from freeing it but if you get it back in Rust you need to then do a Box::from_raw to have Rust manage it again. Although it doesn't look like you're working at this low of a level. :man_shrugging:

If Ruby frees something from memory via the GC but it's still in Rust's scope then that would cause a problem of a double free. In the case where you know Ruby's GC will free the object then you can use Rust's std::mem::forget . I think using this at the end of the scope of the block you need the item to live in will have the same effect as above. Just be sure about which language is calling free.

In fact, it sounds like Ruby's GC is looking for the object after Rust has already freed it. So using std::mem::forget in Rust would let the Ruby GC handle it just fine.

Let me just reiterate that this is me speculating. It might not be the object you've asked about… it might be the thread object itself. So I'll be looking more into it.

dsander commented 5 years ago

Thanks for your response and for looking into the issue!

I tried your suggestions but sadly could not make them work. Please note that I am not at all familiar with unsafe rust so all I am doing is trying to get closer to the root cause while not understanding much what is going on 😄

While digging around I found a simpler reproduction of the problem that does not call any ruby functions:

    fn work2(_input: RString) -> RString {
        let worker_class = "Object".to_string();
        let b = Thread::call_without_gvl(
            move || {
                // Uncommenting this line "fixes" the issue
                // let worker_class = "Object".to_string();
                worker_class
            },
            Some(|| {}),
        );
        return RString::new(&b);
    }

It causes the same pointer being freed was not allocated failure

lldb trace ``` (lldb) target create ~/.rbenv/versions/2.5.3/bin/ruby ^[[ACurrent executable set to '~/.rbenv/versions/2.5.3/bin/ruby' (x86_64). (lldb) breakpoint set --name malloc_error_break Breakpoint 1: where = libsystem_malloc.dylib`malloc_error_break, address = 0x00000000000170de (lldb) run -Ilib fail.rb Process 48310 launched: '/Users/dominik/.rbenv/versions/2.5.3/bin/ruby' (x86_64) ruby(48310,0x1000b95c0) malloc: *** error for object 0x10202f4a0: pointer being freed was not allocated ruby(48310,0x1000b95c0) malloc: *** set a breakpoint in malloc_error_break to debug Process 48310 stopped * thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1 frame #0: 0x00007fff5e9a00de libsystem_malloc.dylib`malloc_error_break libsystem_malloc.dylib`malloc_error_break: -> 0x7fff5e9a00de <+0>: pushq %rbp 0x7fff5e9a00df <+1>: movq %rsp, %rbp 0x7fff5e9a00e2 <+4>: nop 0x7fff5e9a00e3 <+5>: nopl (%rax) Target 0: (ruby) stopped. (lldb) up frame #1: 0x00007fff5e993676 libsystem_malloc.dylib`malloc_vreport + 437 libsystem_malloc.dylib`malloc_vreport: 0x7fff5e993676 <+437>: cmpb $0x1, 0x32c47d6f(%rip) ; debug_mode + 3 0x7fff5e99367d <+444>: jne 0x7fff5e9936a3 ; <+482> 0x7fff5e99367f <+446>: leaq 0x18458(%rip), %rsi ; "*** sending SIGSTOP to help debug\n" 0x7fff5e993686 <+453>: movl $0x5, %edi (lldb) up frame #2: 0x00007fff5e9934a3 libsystem_malloc.dylib`malloc_report + 152 libsystem_malloc.dylib`malloc_report: 0x7fff5e9934a3 <+152>: movq 0x32c45b66(%rip), %rax ; (void *)0x00007fff915c6070: __stack_chk_guard 0x7fff5e9934aa <+159>: movq (%rax), %rax 0x7fff5e9934ad <+162>: cmpq -0x8(%rbp), %rax 0x7fff5e9934b1 <+166>: jne 0x7fff5e9934bc ; <+177> (lldb) up frame #3: 0x000000010250f04a librutie_ruby_example.dylib`alloc::alloc::dealloc::h96e70b1703a1a4e1(ptr="-\t \x10", layout=Layout @ 0x00007ffeefbfed70) at alloc.rs:96 (lldb) up frame #4: 0x000000010250ee66 librutie_ruby_example.dylib`_$LT$alloc..alloc..Global$u20$as$u20$core..alloc..Alloc$GT$::dealloc::h5e1dafd690fb1da6(self=0x00007ffeefbfef88, ptr=NonNull @ 0x00007ffeefbfeda8, layout=Layout @ 0x00007ffeefbfedb0) at alloc.rs:159 (lldb) up frame #5: 0x00000001025048e5 librutie_ruby_example.dylib`_$LT$alloc..raw_vec..RawVec$LT$T$C$$u20$A$GT$$GT$::dealloc_buffer::h489ba15f77bf88dc(self=0x00007ffeefbfef88) at raw_vec.rs:720 (lldb) up frame #6: 0x00000001025078f5 librutie_ruby_example.dylib`_$LT$alloc..raw_vec..RawVec$LT$T$C$$u20$A$GT$$u20$as$u20$core..ops..drop..Drop$GT$::drop::h4b9de7c77d4eb242(self=0x00007ffeefbfef88) at raw_vec.rs:729 (lldb) up frame #7: 0x0000000102503135 librutie_ruby_example.dylib`core::ptr::drop_in_place::hf861c3e4c27dfb43((null)=0x00007ffeefbfef88) at ptr.rs:194 (lldb) up frame #8: 0x0000000102502f62 librutie_ruby_example.dylib`core::ptr::drop_in_place::h42a5b380c5d51a17((null)=0x00007ffeefbfef88) at ptr.rs:194 (lldb) up frame #9: 0x00000001025030c5 librutie_ruby_example.dylib`core::ptr::drop_in_place::hc4a35a77d9ed0296((null)=0x00007ffeefbfef88) at ptr.rs:194 (lldb) up frame #10: 0x00000001025017ae librutie_ruby_example.dylib`rutie_ruby_example::work2::hbbc192169f7fa567(argc=0, argv=0x0000000100600040, _itself=RutieExample @ 0x00007ffeefbfef08) at lib.rs:50 47 Some(|| {}), 48 ); 49 return RString::new(&b); -> 50 } 51 ); 52 53 #[allow(non_snake_case)] (lldb) up frame #11: 0x00000001002d5b34 libruby.2.5.dylib`vm_call_cfunc + 292 libruby.2.5.dylib`vm_call_cfunc: 0x1002d5b34 <+292>: movq %rax, %r13 0x1002d5b37 <+295>: movq (%r15), %rdx 0x1002d5b3a <+298>: addq $0x30, %rdx 0x1002d5b3e <+302>: cmpq %rbx, %rdx ```

The stacktrace reads like rust is dropping a vector? I did not see anything on my or rutie code that allocates a vector or buffer 😕

Because the issue only happens when the closure isn't a "bare" function but captures a variable I assume the cause must lie somewhere in the code that wraps the closure?

danielpclark commented 5 years ago

Reading the documentation here in Rutie I noticed one major difference between what you're doing and what the documentation recommends. You're putting the closures for the thread inside the thread call. I would advise against that. Here's the documentation:

Examples

#[macro_use] extern crate rutie;

use rutie::{Class, Fixnum, Object, Thread};

class!(Calculator);

methods!(
    Calculator,
    itself,

    fn heavy_computation() -> Fixnum {
        let computation = || { 2 * 2 };
        let unblocking_function = || {};

        // release GVL for current thread until `computation` is completed
        let result = Thread::call_without_gvl(
            computation,
            Some(unblocking_function)
        );

         // GVL is re-acquired, we can interact with Ruby-world
        Fixnum::new(result)
    }
);

fn main() {
    Class::new("Calculator", None).define(|itself| {
        itself.def("heavy_computation", heavy_computation);
    });
}

The important lines here are the fact that the closures:

let computation = || { 2 * 2 };
let unblocking_function = || {};

Are outside of Thread::call_without_gvl. Since we're handing Ruby's C the closures via a pointer we don't want them freed within the Thread::call_without_gvl block's scope but after.

Give that a try. Also try it without move.

danielpclark commented 5 years ago

If you're interested in what Ruby has to say about the safety of these methods you can read that in their own source code here: https://github.com/ruby/ruby/blob/ruby_2_6/thread.c#L1453-L1530

Please note that I am not at all familiar with unsafe rust

Surprisingly std::mem::forget isn't unsafe. It just leaves the responsibility of freeing the memory up to you.

dsander commented 5 years ago

You're putting the closures for the thread inside the thread call. I would advise against that. Here's the documentation:

I tried that in my simple example but it does not seem to make a difference.

In fact I am having trouble using call_without_gvl in my real word use case. I basically have a client crate written in rust that I'd like to use to fetch data and make it accessible to ruby. My approach is to initiate the connection, store the client struct in a wrappable_struct! and then use it to make the network call:

let consumer = itself.get_data_mut(&*CONSUMER_WRAPPER);
let handler = || {
    consumer.fetch::<&str>(None)
};
let res = Thread::call_without_gvl(handler, Some(|| {}));

Even if a move closure would not segfault I don't see how I could satisfy the 'static lifetime requirement of the closure :/

When I initiate the client connection in the closure everything works as expected, but initiating a new connection for every call isn't very practical 😄

danielpclark commented 5 years ago

I basically have a client crate written in rust that I'd like to use to fetch data and make it accessible to ruby. My approach is to initiate the connection, store the client struct in a wrappable_struct! and then use it to make the network call

When I initiate the client connection in the closure everything works as expected, but initiating a new connection for every call isn't very practical

I have a couple thoughts on this and it may not be related to the core issue you're having.

Threading

On possibility sounds like the resource of client connecting may not be ready by the time the next thread is trying to use it. One traditional way is to lock the resource while it's used and unlock it for the next thread once it's done with a request. Another is to use Rust Channels which I like as a sort of thread safe queue.

GC

The other thing is since Ruby is trying to free something from memory that has already been freed it would be really nice to know what that is. Each time you use a thread block that itself is an object so that's one possibility.

Now I'm not really sure which language has the responsibility for freeing objects that Ruby uses… but based on experience every method call to the C Ruby API (which Rutie does internally) does things that Ruby takes responsibility for freeing. I know this because if we try to use heap memory rather than stack memory (to hold Ruby objects in Rust) it will cause it to fail.

Now Ruby has a way of marking things that should live longer on not be freed when the GC is called. I'm not sure our GC::mark documentation is meaning to say what it says as it sounds like we “mark objects to be freed” but when I read documentation elsewhere it seems to indicate that instead it's marked for living.

Here's some insight from: https://ruby-hacking-guide.github.io/gc.html

At the moment when there’s not any reachable object left, check all objects in the object pool, release (sweep) all objects that have not marked.

The objects marked here are “objects which are obviously necessary”. In other words, “the roots of GC”

... local variables and arguments of C are automatically marked. For example,

static int f(void) { VALUE arr = rb_ary_new();

    /* …… do various things …… */
}

like this way, we can protect an object just by putting it into a variable. This is a very significant trait of the GC of ruby. Because of this feature, ruby’s extension libraries are insanely easy to write.

If what you're writing is a short lived script you can simply disable the GC as the memory will be freed when the Ruby program exits anyways. This will remedy the memory freeing issue.

danielpclark commented 5 years ago

You could also use GC.disable on the Ruby side of things until all the threads have finished their task and then use GC.enable and GC.start ... just an idea. It's a rough patch of a fix without getting too deep into a solution.

And if the problem still occurs then then you'd need to figure out which item to use std::mem::forget on.

dsander commented 5 years ago

On possibility sounds like the resource of client connecting may not be ready by the time the next thread is trying to use it.

That should not be an issue, I was initiating the connection without releasing the GVL.

Now I'm not really sure which language has the responsibility for freeing objects that Ruby uses… but based on experience every method call to the C Ruby API (which Rutie does internally) does things that Ruby takes responsibility for freeing.

Makes sense to me, I would assume that calling the C API to create an Object behaves the same as calling it from ruby, i.e. the Ruby GC is responsible to free the objects.

but when I read documentation elsewhere it seems to indicate that instead it's marked for living.

I think so too, as far as I know the Ruby GC sweeps "unmarked" object that are not "attached" to any marked object.

Thanks for your input, I think we were looking at the problem from a too high level. I derailed into the problem I am seeing with calling the API because I was not sure if it will work for my use case, but should have focused on the segfault instead.

I did some more digging and am pretty certain that Ruby or the GC does not interfere with the rust/rutie code. All the stack traces I saw were only running rust code (even though it technically calls the ruby C API).

Below are a few experiments I made which helped me better understand when the issue happens, but sadly not why.

The examples are a bit verbose, but I think they show two issues. Capturing stack allocated variables does not work correctly (stack_allocated_returning_input/stack_allocated_returning_from_closure), something in thread_call_callbox or utils:closure_to_ptr must mess up the pointer (this is not related to my original issue).

Returning "simple" types from the closure seems to work (I am assuming a usize allocated during runtime does not live on the stack?).

My original issue happens in heap_allocated_returning_input and heap_allocated_returning_from_closure. Here the value of the variable is capture correctly in the closure, but when it's returned the pointer points at the wrong location which then causes the pointer being freed was not allocated errors.

The full source code is here: https://github.com/dsander/rutie/tree/gvl-debugging

fn heap_allocated_returning_input() -> NilClass {
    let input = "Object".to_string();
    let handler = move || {
        // Trying to make sure the compiler doesn't optimize the whole function away
        let metadata = File::open("/Users/dominik/code/rust/rutie/examples/rutie_ruby_example/fail.rb").unwrap().metadata().unwrap();
        metadata.len();
        println!("inside closure: '{}'", input);
        input
    };
    let ret = Thread::call_without_gvl(handler, Some(|| {}));
    println!("returned value: '{}'", ret);
    return NilClass::new();
}
// Output:
// inside closure: 'Object'
// returned value: '���c�'
// ruby(23643,0x115ba85c0) malloc: *** error for object 0x7ff63ca123d0: pointer being freed was not allocated
// ruby(23643,0x115ba85c0) malloc: *** set a breakpoint in malloc_error_break to debug
//
// Somehow the pointer to the string gets messed up when returned from the closure, after `ret` gets
// out of scope rust tried to deallocate some object that does not exist (at least not at the
// address that is freed)

fn stack_allocated_returning_input() -> NilClass {
    let input = 42;
    let handler = move || {
        // Trying to make sure the compiler doesn't optimize the whole function away
        let metadata = File::open("/Users/dominik/code/rust/rutie/examples/rutie_ruby_example/fail.rb").unwrap().metadata().unwrap();
        metadata.len();
        println!("inside closure: '{}'", input);
        input
    };
    let ret = Thread::call_without_gvl(handler, Some(|| {}));
    println!("returned value: '{}'", ret);
    return NilClass::new();
}
// Output:
// inside closure: '-759049584'
// returned value: '-759049584'
//
// Similar behavior as in `heap_allocated_returning_input` but since nothing needs to be freed
// we just get a random value from somewhere on the stack(?)

fn heap_allocated_returning_from_closure() -> NilClass {
    let input = "Object".to_string();
    let handler = move || {
        // Trying to make sure the compiler doesn't optimize the whole function away
        let metadata = File::open("/Users/dominik/code/rust/rutie/examples/rutie_ruby_example/fail.rb").unwrap().metadata().unwrap();
        println!("inside closure: '{}'", input);
        metadata.len()
    };
    let ret = Thread::call_without_gvl(handler, Some(|| {}));
    println!("returned value: '{}'", ret);
    return NilClass::new();
}
// Output:
// inside closure: 'Object'
// ruby(25217,0x1016035c0) malloc: *** error for object 0x7fe46e51d6c0: pointer being freed was not allocated
// ruby(25217,0x1016035c0) malloc: *** set a breakpoint in malloc_error_break to debug
//
// The free is called somewhere in `util::ptr_to_data`, I assume it is trying to free the
// `input` before returning the result from the closure (which makes sense). However I don't
// understand why the free is called for the wrong pointer as the value of `input` inside the
// closure looks correct.

fn stack_allocated_returning_from_closure() -> NilClass {
    let input = 42;
    let handler = move || {
        // Trying to make sure the compiler doesn't optimize the whole function away
        let metadata = File::open("/Users/dominik/code/rust/rutie/examples/rutie_ruby_example/fail.rb").unwrap().metadata().unwrap();
        println!("inside closure: '{}'", input);
        metadata.len()
    };
    let ret = Thread::call_without_gvl(handler, Some(|| {}));
    println!("returned value: '{}'", ret);
    return NilClass::new();
}
// Output:
// inside closure: '623171904'
// returned value: '79'
//
// Again the input is not moved correctly to the closure but returning from the closure works as
// expected.
danielpclark commented 5 years ago

I have a hunch. Now I'm no expert with either threads or C pointers… I've only been learning C for this project as I've pretty much inherited it. But I think it may have something to do with the first parameter to the Ruby C API for the thread methods being this void *(*func)(void *).

This signature breaks down as:

_Each pointer could be translated to *const c_void or *mut c_void as a Rust FFI method signature and typically work._

I came across this Rust issue Pointer-to-pointer + casts in FFI leads to hard-to-debug mysterious problems.

Now I could be wrong for many reasons, but here are my thoughts.

The issue shows this method extern "C" fn c_func(x: *mut *mut libc::c_void); should be called as (&mut x) as *mut _ as *mut *mut libc::c_void where the underscore is simply an intermediate cast the compiler figures out on its own.

Looking at util::closure_to_ptr the function provided gets returned as Box::into_raw(Box::new(fnbox)) as *const c_void and Box::into_raw outputs *mut T. So the util::closure_to_ptr is kind of saying *mut T as *const c_void. But the Ruby C method of a pointer to a pointer would lead me to believe the Rust side of things needs either a borrow signature at the beginning (the (&mut x) from the issue), or to double down on the pointer return type (the as *mut *mut c_void part), or both.

Whatever the case may be I think it would be wise to add a code example which runs thread tests in the root examples/ folder which will be tested on CI each time we submit changes… just as the current eval example gets tested.

Trying out your current fork, it has an absolute path for the file metadata that only works on your system and the location and use of fail.rb is a bit unconventional for testing: eg using cargo build --release; bundle exec ruby fail.rb rather than rake test. If you want to I'd be okay with a examples/rutie_ruby_thread_example folder, which won't run on CI, but can be tried by others.

danielpclark commented 5 years ago

Oops… I misread the function

let ptr = thread::rb_thread_call_with_gvl(
    thread_call_callbox as CallbackPtr,
    util::closure_to_ptr(func),
);

util::ptr_to_data(ptr)

It's using thread_call_callbox as CallbackPtr so the util::closure_to_ptr is not the area to look at.

Useful FFI info can be found here: https://areweextendingyet.github.io/ and the most informative resource is: https://doc.rust-lang.org/nomicon/ffi.html and additional info can be found in the Rust forum: https://users.rust-lang.org/