representing a `Box<T>` with a custom block

[c-cube]

I have some bindings I'd like to upgrade, but they used some unsafe pointer mangling and Value::alloc_custom(ptr, finalizer) to build OCaml values. What's the new idiom for that, can I just say return Box<MyStruct> and have the #[ocaml::func] macro take care of the rest?

[zshipko]

I really like the idea of automatically handing Box<T>. I will try to see if I can figure out how to do that, though I'm not sure how to preserve the generic type across an extern "C" call.

Currently the way to handle this is using Value::alloc_final and a finalizer. See https://github.com/zshipko/ocaml-vec/blob/master/src/lib.rs#L3

[zshipko]

I've also added Value::alloc_custom which allows you to create a proper custom type to write comparison and hash functions, which is the recommended way of doing this. But the above answer is closest to your current implementation.

[zshipko]

Just added a little more information to the README: https://github.com/zshipko/ocaml-rs#pointers-to-rust-values-on-the-ocaml-heap

Closing this, but I will make sure to include information about Pointers and custom values in a more complete tutorial soon.

[progman1]

I was about to tack on a question to my closed issue when I noticed this and the mention of Box. I have a function that returns a Box and while no problem packing that into a Pointer I have not stumbled onto a way to extract such for the consuming function that takes in this Box (after passing through OCaml). your comment above prompts me to ask for a way to do this non-automatically in the interim. I imagine doing such a thing would mean the Box becomes invalid in the OCaml world, in which case I would then need to arrange a monadic wrapper to ensure single-use of said Box.

incidentally, why do you use drop-in-place() over drop()?

[zshipko]

You can use Pointer::as_ref or Pointer::as_mut to get a reference to the inner value. A Pointer should be used any time you want to pass opaque or Custom Rust values into the OCaml.

If you can ensure that your boxed value lives long enough then you can pass it into OCaml without allocating it on the OCaml heap, but this is bound to cause lifetime issues unless you’re very careful.

drop_in_place is used here because the memory is allocated by OCaml, so after drop_in_place is called and the Rust value is dropped the OCaml runtime will clean up the allocated memory. After trying some other methods, this seemed to be the safest way of handling this.

[c-cube]

I think you cannot move out of a raw pointer, so you have to use ptr::drop_in_place to drop the value through the pointer.

[progman1]

Zach the issue I'm having is extracting a Box from the Pointer with ownership because that is what the target rust function requires. as_ref/as_ptr/as_mut don't seem suitable: o as_ptr because compiler reports inability to 'move out of a raw pointer' o as_ref because it's providing a borrow and not ownership

I'm just finding my feet wrt rust and I don't know whether this is a limitation of your interface or a fundamental restriction!

c-cube I see. thanks!

On 09/05/2020, Simon Cruanes notifications@github.com wrote:

I think you cannot move out of a raw pointer, so you have to use ptr::drop_in_place to drop the value through the pointer.

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/zshipko/ocaml-rs/issues/28#issuecomment-626182343

[zshipko]

Without a code example, I'm not 100% sure what your use case is. Getting a Box<T> out of a Pointer<T> means you would be passing ownership of the pointer to Rust, but Pointer is typically used for values on the OCaml heap, which means OCaml may choose to garbage collect it at any time. So, I think this is a purposeful limitation of the interface.

If you have a Pointer<Box<T>> there might be a way to get around that limitation, I will need to think about it though since this isn't something I've considered. Another option could be to clone the value, using something like my_pointer.as_ref().clone() but that is making a copy which may not be ideal.

[progman1]

the thing stored in a Pointer is a Box and the consuming function for it, on the rust side, requires Box<dyn X> where X is the trait Y implements. so I'm trying to convince the compiler that the pointer stored in the ocaml custom block is in fact a trait object (a double ptr struct ?) but cannot cast to a Box as not considered 'primitive'.

On 09/05/2020, zach notifications@github.com wrote:

Without a code example, I'm not 100% sure what your use case is. Getting a Box<T> out of a Pointer<T> means you would be passing ownership of the pointer to Rust, but Pointer is typically used for values on the OCaml heap, which means OCaml may choose to garbage collect it at any time. So, I think this is a purposeful limitation of the interface.

If you have a Pointer<Box<T>> there might be a way to get around that limitation, I will need to think about it though since this isn't something I've considered. Another option could be to clone the value, using something like my_pointer.as_ref().clone() but that is making a copy which may not be ideal.

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/zshipko/ocaml-rs/issues/28#issuecomment-626197355

[zshipko]

How is the Pointer<Box<Y> allocated? If you're using alloc_custom or alloc_final then it's not really safe to access that inner Box<Y> value directly.

Otherwise if you're allocating the value on the Rust side and passing it into OCaml it is prbably possible, but not something I've been focusing on.

Another option would be to implement trait X for &Y, then you can use Box::new(ptr.as_ref()) or something like that to pass to the other function.

Or it looks like you may be able to use std::ptr::read (https://doc.rust-lang.org/std/ptr/fn.read.html)

[progman1]

I'm over the hurdle!: the rust value I had to store was a Box thus double pointer size. so when I tried *v.as_ptr() to get it back out of the Pointer it was non-copyable. using ManuallyDrop on it allowed a (v.as_ref()).take() to do the extraction of the box.

now a possible bug report: I have a 5 arg function that became 6: pub fn neuro_model_fit( mut net: ONetwork, data: OTabularDataSet, batch_size: u64, epochs: u64, print_loss: Option,

which then yielded the error:

error: mut on a binding may not be repeated --> src/lib.rs:333:2 | 333 | mut net: ONetwork, | ^^^ help: remove the additional muts

so perhaps this is to do with the attribute macro switching to >5 args?

On 10/05/2020, zach notifications@github.com wrote:

How is the Pointer<Box<Y> allocated? If you're using alloc_custom or alloc_final then it's not really safe to access that inner Box<Y> value directly.

Otherwise if you're allocating the value on the Rust side and passing it into OCaml it is prbably possible, but not something I've been focusing on.

Another option would be to implement trait X for &Y, then you can use Box::new(ptr.as_ref()) or something like that to pass to the other function.

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/zshipko/ocaml-rs/issues/28#issuecomment-626350880

[zshipko]

@progman1 Thanks, this is most likely a bug in one of the ocaml function macros. I will try to figure out what's going on here.

[zshipko]

@progman just published a new version of ocaml-derive - if that doesn't fix your problem please open a new issue.