Hi! I'm an engineer at Microsoft working on the Babylon Native project (GitHub repo here). We'd like to create a plugin that will allow developers to easily incorporate Babylon.js-powered 3D rendering directly into their React Native apps, preferably keeping the integration as seamless as possible.
Babylon Native works by running Babylon.js inside a JavaScript context, much like React Native does. Ideally, we would like our plugin to function by making Babylon.js types directly available to React Native developers inside their existing JavaScript context, with the native-level integration that drives the rendering handled behind the scenes. The trick, however, is that Babylon Native uses Node.js's C++ API to talk to JavaScript, and we'd like to avoid taking a direct dependency on abstractions specific to React Native. We've come up with a working (though slightly hack-ish) prototype, and we're evaluating ways to accomplish the aforementioned goals for Babylon Native as well as any other plugins attempting something similar.
The Core of It
The core thing we want to enable is the integration of native-level libraries that already do depend on a JavaScript runtime but don't depend directly on React Native. In the case of Babylon Native, for example, the library (at present) is able to communicate directly with JavaScriptCore, but has no knowledge of React Native concepts such as JSI. If, however, we are able to get a very small amount of low-level access to the JavaScript runtime itself -- not the JSI abstraction, but the underlying JavaScript context -- then Babylon Native and similiarly-structured libraries will be able to communicate with the JavaScript in their own way, without requiring the core library to have a direct dependency on React Native.
Discussion points
As I mentioned above, we hacked together a working prototype just to help us understand the issues involved, and that prototype ended up proving out one possible, reasonably non-invasive way this information might be exposed. In our prototype, we modified JSCRuntime.h and JSCRuntime.cpp to expose both the JSCRuntime type and the getContext method. We then built a native (C++) Android plugin against that header and binary which reinterpret_casts a void* to a JSCRuntime*, then calls the exposed getContext method to get the JSGlobalContextRef. For our prototype, that was enough for us to build a very simple Napi::Env using Babylon Native's N-API abstraction for JavaScriptCore, then use that to create an object from C++ on the JavaScript global object. Inside the Android app, we provided the void* pointer to the JSCRuntime by retrieving it from the ReactContext and passing it down from the Java layer. Once that was done, we were able to check from the React Native JavaScript code and confirm that the object we'd created in C++ using N-API was indeed available in the familiar React Native JS environment.
Lots of hacks in that proof of concept, but it did help prove out that we can open the door for other native-level JavaScript integrations by exposing out only a few bits of information about the underlying JavaScript runtime. In our case, we were able to create a small object using only the JSGlobalContextRef, though we almost certainly ended up creating that object on the wrong thread. Thus, for JavaScriptCore specifically, we think the only two things we'd need exposed in order to be able to integrate Babylon Native are (1) the JSGlobalContextRef and (2) a mechanism with which to run code on the JavaScript thread, like a RunOnJavaScript() method or something like that.
To be clear, we have an idea that we believe could work, but we are very early in this thought process, so please jump in with whatever thoughts, questions, or concerns you think are relevant. Some of the questions that are already top-of-mind for us are as follows:
Is the mechanism discussed above (the context pointer and run-on-thread method) the right way to tackle this problem? If so, what's the best way to expose this mechanism?
How can we avoid forcing libraries to take a dependency if all they want to share is the JavaScript context? To avoid direct calls and types, perhaps the context pointer and run-on-thread method (and other information as needed) could be passed down from the app as void*, and the consuming library is simply required to already know what they are (in the described case, a JSGlobalContextRef and a function pointer, respectively)? Is there a better way?
Are these two capabilities enough? Are there other libraries that might want to attempt something similar which need other things to be exposed? Could a different JavaScript runtime, like v8 or Hermes, be exposed in the same way (assuming the consuming library was already aware of the JavaScript runtime in question)?
(Edited to refer to JSI instead of JNI, which I'd accidentally written at first.)
shergin
commented
4 years ago
Introduction
Hi! I'm an engineer at Microsoft working on the Babylon Native project (GitHub repo here). We'd like to create a plugin that will allow developers to easily incorporate Babylon.js-powered 3D rendering directly into their React Native apps, preferably keeping the integration as seamless as possible.
Babylon Native works by running Babylon.js inside a JavaScript context, much like React Native does. Ideally, we would like our plugin to function by making Babylon.js types directly available to React Native developers inside their existing JavaScript context, with the native-level integration that drives the rendering handled behind the scenes. The trick, however, is that Babylon Native uses Node.js's C++ API to talk to JavaScript, and we'd like to avoid taking a direct dependency on abstractions specific to React Native. We've come up with a working (though slightly hack-ish) prototype, and we're evaluating ways to accomplish the aforementioned goals for Babylon Native as well as any other plugins attempting something similar.
The Core of It
The core thing we want to enable is the integration of native-level libraries that already do depend on a JavaScript runtime but don't depend directly on React Native. In the case of Babylon Native, for example, the library (at present) is able to communicate directly with JavaScriptCore, but has no knowledge of React Native concepts such as JSI. If, however, we are able to get a very small amount of low-level access to the JavaScript runtime itself -- not the JSI abstraction, but the underlying JavaScript context -- then Babylon Native and similiarly-structured libraries will be able to communicate with the JavaScript in their own way, without requiring the core library to have a direct dependency on React Native.
Discussion points
As I mentioned above, we hacked together a working prototype just to help us understand the issues involved, and that prototype ended up proving out one possible, reasonably non-invasive way this information might be exposed. In our prototype, we modified JSCRuntime.h and JSCRuntime.cpp to expose both the
JSCRuntimetype and thegetContextmethod. We then built a native (C++) Android plugin against that header and binary whichreinterpret_casts avoid*to aJSCRuntime*, then calls the exposedgetContextmethod to get theJSGlobalContextRef. For our prototype, that was enough for us to build a very simpleNapi::Envusing Babylon Native's N-API abstraction for JavaScriptCore, then use that to create an object from C++ on the JavaScript global object. Inside the Android app, we provided thevoid*pointer to theJSCRuntimeby retrieving it from theReactContextand passing it down from the Java layer. Once that was done, we were able to check from the React Native JavaScript code and confirm that the object we'd created in C++ using N-API was indeed available in the familiar React Native JS environment.Lots of hacks in that proof of concept, but it did help prove out that we can open the door for other native-level JavaScript integrations by exposing out only a few bits of information about the underlying JavaScript runtime. In our case, we were able to create a small object using only the
JSGlobalContextRef, though we almost certainly ended up creating that object on the wrong thread. Thus, for JavaScriptCore specifically, we think the only two things we'd need exposed in order to be able to integrate Babylon Native are (1) theJSGlobalContextRefand (2) a mechanism with which to run code on the JavaScript thread, like aRunOnJavaScript()method or something like that.To be clear, we have an idea that we believe could work, but we are very early in this thought process, so please jump in with whatever thoughts, questions, or concerns you think are relevant. Some of the questions that are already top-of-mind for us are as follows:
void*, and the consuming library is simply required to already know what they are (in the described case, aJSGlobalContextRefand a function pointer, respectively)? Is there a better way?(Edited to refer to JSI instead of JNI, which I'd accidentally written at first.)