shared/transfered object ownership

[sebastiang]

It would be great to be able to return ownership of an object (like a string or image buffer) from a Julia function to the V8 engine to avoid copying, or for the Julia function to work on a V8 owned object (like a TypedArray). I would be happy to contribute to writing the code if there are pointers in the right direction.

[waTeim]

Oh man, you're right, that would be great! Did you seen my initial comment in the ANN thread on this? Those comments are generally correct still, but more specifically.

There's a mapping made for convenience, and multi-dimensional arrays are going to be a big pain when doing this because Julia multidimensional array elements are all contiguous and column-major while neither is true with Javascript, so probably best do single dimensional arrays to start with.

The V8 interface between version 10 and 12 that allows TypedArray allocation changed. I'm still pretty sure it's possible in 10, but it's pretty obvious what to do in 12. There are 2 files nj-v10.cpp and nj-v11.cpp, and rather than trying to re-factor like crazy, I simply did it twice after some initial cut/paste. As it turns out, I think that was and remains the easier route. That 2nd file doesn't have the best name anymore.

So the critical section of code in nj-v11 is here

currently the ArrayBuffer backing store is initialized this way

Local<ArrayBuffer> buffer = ArrayBuffer::New(I,size0*sizeof(Nv));

But it need not be initialized that way, but instead (as described here),

Local<ArrayBuffer> buffer = ArrayBuffer::New(I,array.ptr(),size0*sizeof(Nv));

then the julia buffer will be used instead, and no new allocation will happen. So that one liner is pretty easy, however, now v8 will not free the data when the TypedArray is garbage collected, and Julia doesn't know either. What is really needed is some sort of message that is sent. It took a while to find it, but I think there's this thread

Since V8 does not know how the memory backing an ArrayBuffer has been allocated in this case, there is now good way for V8 to free it. Just like previously with "external indexed data", the way to do it is to use a weak persistent handle for the ArrayBuffer to get a notification when ArrayBuffer becomes unreachable and then deal with your memory block accordingly.

The Persistent API is documented here. When you do MakeWeak, it allows for a callback and some parameters (see below).

Meanwhile, Julia needs to also not GC the array or image until Javascript is done with it. That's already done with JRefs and implemented here. So an approach is to associate some Id with the ArrayBuffer and then later in the callback check the id and free it.

That should get you started, I probably left out some details and then there's the 0.10 implementation to think about, or maybe just start with use node 12, though alot of people are still using 10 so it's desirable.

Thanks much in advance!

[waTeim]

An update on this one; with d7f308ac26d, the space for a single dimension Javascript Typed Array is now allocated when creating the lvalue only rather than both the lvalue and again during Array::New see here. A weak reference is created at the same time, and then the buffer is cleaned up when the V8 GC runs. There is more to do, but this is the general pattern.

[waTeim]

The reverse mapping problem is now addressed as well. Only one more thing remains; expressions that evaluate to the same jl_value_t should also map to the same Javascript object. This can be straightforwardly addressed with a jl_value_t to JuAlloc map followed by a check for a related JSAlloc through their shared container.