search

react-native-community / discussions-and-proposals

Discussions and proposals related to the main React Native project
https://reactnative.dev
1.66k stars 126 forks source link

TurboModules (NativeModules Re-architecture) #40

Closed fkgozali closed 2 years ago

fkgozali commented 5 years ago

Introduction

This is a place for discussions around the upcoming "TurboModule" feature.

Terminology

TL;DR

From @axe-fb's blogpost, here's a temporary description of TurboModules (please consider that this is not yet finalized, it may change in the future)

The JSI system can also be used to call leverage device capabilities like bluetooth or other sensors by exposing functions that JS can call. This is similar to how browsers expose functions like navigator.geolocation.getCurrentPosition that, when invoked in JavaScript, trigger the respective C++ call in the browser. In the current system, a table with information about module names and methods is created. When JS calls a specific native module, the indices of the module and methods are passed to Java/ObjC, which then invoke the specific methods. The arguments and return values are also converted between JavaScript and JNI/ObjC objects. [...] Now that we have a JSI object for "SampleTurboModule", can invoke methods on this JSI object from JavaScript. During the calls, we also need to convert JSI Values to JNI for argument parameters, and the reverse when sending back results. Like in the current architecture, most types including boolean, strings, Maps, Arrays, Callbacks and Promises are supported.

Available Materials

At ReactConf 2018 @axe-fb did a talk about React Native's New Architecture, which also explains the 3 concepts above: JSI, Fabric, TurboModule.

IN Q1 2019, @kelset wrote a more high-level explanation in a blogpost: https://formidable.com/blog/2019/fabric-turbomodules-part-3/ and did a talk about the whole rearchitecture in April 2019 at React Edinburgh.

@kelset also did a more in-depth talk at React Advanced London in Oct 2019: youtube recording & slides.

Q&A

This is also a place for questions related to this effort and its direction.

wqsflying commented 4 years ago

Hey @sav007, you can use Java/Kotlin as you would use before. Turbomodules aren't about rewriting everything to C++, they are about several things:

  1. Lazy initialization – allows you to initialize module instance only then you calling it at the very first time, not on the app initialization itself.
  2. JSI – JavaScript Interface – in fact, this is the new iteration of the "bridge" helping you triggering all the Java code from the JS. The old bridge has overhead – JSON serialization of each call on both sides – from the JS and from the Java. But JSI allows you to skip that part, so it's much faster.
  3. JSI is completely synchronous.

So you will need to create a module spec and "codegen" the C++ part of it, it will create a JSI "bridge" for your module in order to allow you to call all your methods from JS really easy and get response blazingly fast.

great

mpiannucci commented 4 years ago

So I tried to make my own TurboModule and Fabric Module the other day. I could already make a JSI C++ module no problem, so I wanted to try a real turbo module next. The only missing piece at this piece seems to be the codegen. It is in master and I am able to track how react native calling it when building for iOS, but could not really figure out how I would use it with a new module not in the react native core.

Anyways, just wanted to report that it seems to be really close and I am pumped (as a C++ dev by day) to start using them.

gillpeacegood commented 4 years ago

I have tried experimenting with turbo module, I am interested in developing a pure c++ turbo module, but wondering how to handle async elements.

I took the following steps :-

  1. checked out react-native master
  2. built the RNTester app documented here :- https://github.com/facebook/react-native/tree/master/RNTester (taking care with the version of cocoapods used)
  3. This includes a test called “TurboModule”
  4. The test TurboModule is an objective c module, but there is a sample C++ module which uses JSI, and you can enable it by changing this loc https://github.com/facebook/react-native/blob/fa9ff07017edbc76595fe2f2d964ee13c5f4088a/Libraries/TurboModule/samples/NativeSampleTurboModule.js#L35 to say ‘SampleTurboCxxModule’
  5. The cxx (= c++) module is here https://github.com/facebook/react-native/tree/master/ReactCommon/turbomodule/samples whereas the objective c one is in platform/ios subdir

However the promise example does block the UI, and I am wondering if anyone can advise a non blocking approach? (I tried this which doesn't entirely work and I am pretty sure its wrong)

 jsi::Value SampleTurboCxxModule::getValueWithPromise(
    jsi::Runtime &rt,
    bool error) {
  return createPromiseAsJSIValue(
      rt, [error, this](jsi::Runtime &rt2, std::shared_ptr<Promise> promise) {
      std::thread([=, &rt2](  ) {
        std::this_thread::sleep_for(std::chrono::seconds(5));
        if (error) {
          promise->reject("intentional promise rejection");
        } else {
          promise->resolve(jsi::String::createFromUtf8(rt2, "result!"));
        }
      }).detach();
  });
}
thegamenicorus commented 4 years ago

Hi guys,

if you need to play with TurboModule and CodeGen outside react-native master. Check out my repo: https://github.com/thegamenicorus/TurboModulePlayground

kelset commented 4 years ago

Just FYI, @stmoy has opened a dedicated conversation to discuss how developers are expected/should expect to start using TurboModules -> https://github.com/react-native-community/discussions-and-proposals/issues/195

I'm sure he'd love to hear your experiences & ideas (since quite a few of you already tried them out!)

DomiR commented 4 years ago

I just saw https://github.com/facebook/react-native/commit/96fdaa541e30e1f52a4649e747f2372bac28b4cf and similar diffs. Why isn't caching constants in JS done more generically (e.g. within the TurboModuleRegistry class), instead of re-implementing the "spec" like done in the diffs? It seems to me that we are introducing the whole boilerplate from the spec just to cache the constants... if this is a good practice, why not do it for everyone?

fkgozali commented 4 years ago

@DomiR

Why isn't caching constants in JS done more generically (e.g. within the TurboModuleRegistry class), instead of re-implementing the "spec" like done in the diffs? It seems to me that we are introducing the whole boilerplate from the spec just to cache the constants... if this is a good practice, why not do it for everyone?

  1. This is NOT a good practice. TurboModule no longer special-case getConstants() unlike the old NativeModules. Which is why we'll be moving away from getConstants() specifics, like general caching in the old system. This is for simplicity and better overall performance.
  2. However, we're trying to isolate some regression that may be caused by the modules affected in the commit you linked. If we can pinpoint the problematic module, it would be easier to investigate the infra fix for it, or tweak the module impl to fix the issue. Doing it for all modules will make it impossible to isolate the issue.
  3. In the future, @RSNara has some plan to allow per-method memoization with simple annotation in the spec, but not for TurboModule v1.
  4. Finally, stay tuned for proper documentation explaining all these details (when it's closer to release time later this year).
mrousavy commented 4 years ago

I kinda lost track of the current state of all of this - Is it already possible to write TurboModules or do we have to wait for the whole react-native re-architecture (including JSI, Fabric, ...)? I can't find a lot of examples, there is this guide that somehow explains how to create a TurboModule in Android, but I haven't tried it out yet. Also, I couldn't find anything for iOS, especially not for Swift.

I'm currently writing a native module that could benefit from a performance boost using TurboModules, also I could write a quick cheat sheet for migrating from native modules, or TurboModules in Swift all together.

DomiR commented 4 years ago

The current state is that it mostly works but we have not seen yet any documentation and no official announcement. I guess we need to still be more patient. I don't know why the react-native team won't do a "is-it-ready-yet" page like the react team did for fiber but I guess they are into big reveals now, as with the announcement of hooks.

But is also a good strategy to wait for a somewhat stable API (adaption will be a bit slower than usual, because it is a new feature and not something "under the hood" as it was the case with fiber, so making sure that lib devs only need to write new code once is probably the right way to go). Of course, it is also an expectation management strategy, as any missed deadlines will be aggressively pointed out by unruly Twitter users.

The good news is, that there are some mentions, that Facebook is already dogfooding TurboModules internally and they already said multiple times that they plan to release it later this year (it was mid of this year before corona). React-reanimated v2 is already written as a TurboModule, but it takes some extra steps to get it running. They've implemented some workarounds and will probably have to re-implement some parts as soon as the official release is here, but it's an early alpha you can check out for reference.

terrysahaidak commented 4 years ago

I still think the main reason it hasn't been released yet is the fact that there is no solution fo remote debugging. As soon as you enable turbo modules, debugger is not available. The only way to debug is it connect to remote context. And it's way far from best debugging experience.

The current solution for this I guess is to enable Hermes in dev on iOS. It supports chrome debugger protocol so it should be easy to integrate existing debugger (as well as VSCode one) but the whole thing will be executed on the device/simulator which allows us to have access to JSI based stuff.

mrousavy commented 4 years ago

@DomiR let's hope they create good guides for TurboModules, because let's be honest, I didn't get much information from the current NativeModules guides on reactnative.dev. My only source of information were 6 month old medium articles.

I've thought about contributing to the docs and rewriting the Native Modules sections to be more detailled, but I'm afraid once TurboModules launch all of this will just be wasted time since NativeModules will be "deprecated" sooner or later anyways.

kelset commented 4 years ago

hey folks - there's no updated at this point in time around TurboModules. I am aware that FB is cooking up something for the next 0.64 release that will be related to that.

I don't want to give you false promises or misinformations, but what I know is that TurboModules before end of 2020 is still the plan on their side.

Enigma10 commented 4 years ago

@kelset Sorry for asking again. But i was checking 0.64 release issue and there is no mention of turbomodule or fiber.

terrysahaidak commented 4 years ago

The Turbomodules, as well as JSI, is pretty much working and you can use it even right now – a great example of this is the new version of Reanimated which uses both JSI and Turbomodules to provide the best possible performance.

But all the JSI bindings have been written by hand which is hard. That's why Codegen is that important in order to get to the point where we use Turbomodules for all the libraries and it's stable. But as you can see in the commit history there are lots of commits related to add support for Codegen for android. The iOS part seems to be ready.

Also, there are constant commits of Fabric related things as well.

There is no ETA but everything is in active development AFAIK.

kelset commented 4 years ago

+1 on what @terrysahaidak is saying (thanks for answer that, I was on holiday last week).

Just a correction on my last comment, the

I am aware that FB is cooking up something for the next 0.64 release that will be related to that.

Was scraped so I don't think in 0.64 there's going to be any TurboModule specific feature drop.

a-eid commented 3 years ago

@terrysahaidak @kelset I'm wondering how will the Animated Api when Turbomodules & jsi are fully out, and how would it compare to reanimated 1 and reanimated 2 ?

terrysahaidak commented 3 years ago

@terrysahaidak @kelset I'm wondering how will the Animated Api when Turbomodules & jsi are fully out, and how would it compare to reanimated 1 and reanimated 2 ?

Nothing really changes for Animated API. Right now (alpha.7 version) you can see in Reanimated 2 installation instruction how to enable TurboModules. After that all the React Native modules (at least on iOS) will use TurboModules and JSI.

But nothing changes because Reanimated 1 and 2 will allow you to write custom animation logic that will be executed on the UI thread directly, just like useNativeDriver does.

a-eid commented 3 years ago

@terrysahaidak thank you, afaik, JSI & TurboModules will eliminate using the RN bridge, which is the main limitation for the Animated Api, any why SWM had to create their awesome RNGH & Reanimated.

will animated still be inefficient to be used compared to reaniamted 1 or 2 ?

terrysahaidak commented 3 years ago

@terrysahaidak thank you, afaik, JSI & TurboModules will eliminate using the RN bridge, which is the main limitation for the Animated Api, any why SWM had to create their awesome RNGH & Reanimated.

will animated still be inefficient to be used compared to reaniamted 1 or 2 ?

Not really. The main benefit of Reanimated - code is executed on UI Thread. Even with sync access to style updating from JS Thread, you still can just block the thread by some expensive business logic or for example when you render a new screen. The same thing may happen on web - that's why css3 transitions is used for such things.

a-eid commented 3 years ago

@terrysahaidak thanks, but still jsi & turbomodules would bring a lot of benefits to the Animated Api.

kelset commented 3 years ago

I'm wondering how will the Animated Api when Turbomodules & jsi are fully out, and how would it compare to reanimated 1 and reanimated 2 ?

Too early to say.

afaik, JSI & TurboModules will eliminate using the RN bridge,

The removal of the bridge will happen at a later stage. The bridge will be around for a while even after JSI + TurboModules + Fabric will be fully rolled out to ensure that everyone can smoothly transition.

axemclion commented 3 years ago

@kelset When the re architecture is completed, do you think that the performance gap with flutter will be filled?

Could you elaborate on the performance gap you refer to, in your question ? Are you talking about startup, TTI, or simply calling native calls ? FWIW, you can use RN plugins with Flutter - http://blog.nparashuram.com/2018/12/using-react-natives-plugins-with.html, and the calling mechanism is similar.

axemclion commented 3 years ago

I'm wondering how will the Animated Api when Turbomodules & jsi are fully out, and how would it compare to reanimated 1 and reanimated 2 ?

Reanimated is actually an excellent example of how JSI is used. At a high level, jumping back and forth between Native to JS would be much simpler. I am also hoping that we are able to use JSI to make interruptable or JS Controllable animated APIs.

byteab commented 3 years ago

@axemclion reanimated not only benefit the jsi but it also run the javascript part of the animation on the UI thread

axemclion commented 3 years ago

@axemclion reanimated not only benefit the jsi but it also run the javascript part of the animation on the UI thread

Are you saying that JSC or Hermes would run on the UI thread ? Could you elaborate what you mean by running JS on the UI thread ? If I understand TurboModules correctly, you would basically "interrupt" UI thread, and wait for a response to the curve function from JS, and then resume the UI thread. Similar to how android animations work, where you have a separate function that helps you calculate the curve.

axemclion commented 3 years ago

@axemclion for example the use of cpu.

It would help to define what you mean by CPU here. Are you refering to the various threads ? For example, if you run JS with Hermes, you would still use a separate thread which will still run the Hermes bytecode. This would be no different than what happens today.

byteab commented 3 years ago

@axemclion Reanimated 2 will spawn a secondary JS context on the UI thread that then is able to run JavaScript functions. have a look on worklet in reanimated 2 doc

kelset commented 3 years ago

@lorenzoangelini it looks like you are not backing your claims with data and without those it's really hard to understand the actual differences you seem to imply there are 🤷‍♂️

EDIT: for future memory, this conversation looks like it doesn't make much sense because the original comment from the author quoted above disappeared (maybe he canceled it).

aanah0 commented 3 years ago

Does it have any chance to support remote debugging in Chrome like it currently working? I really like reanimate 2 library, it awesome. but large debugging is required for large projects

halaei commented 3 years ago

In case you are working on more data-type supports can you consider the followings as well?

  1. Int64 and UInt64 and other primitive data-types (https://developer.mozilla.org/en-US/docs/Mozilla/js-ctypes/js-ctypes_reference/Int64).
  2. Typed arrays (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Typed_arrays).
  3. Blob objects with actual data possibly in native side (https://developer.mozilla.org/en-US/docs/Web/API/Blob), with networking support.
  4. Null character in strings (https://github.com/facebook/react-native/issues/24129)
Luckygirlllll commented 3 years ago

When approximately these TurboModules will be ready?

mrousavy commented 3 years ago

To support swift, such c++ to swift binding needs to be built (we won’t be building this as we’re focused on objc support right now).

Since Swift is already the mainstream language for iOS development (and is easier to understand than ObjC, for JS developers at least) it would make sense to provide first class support for creating TurboModules in Swift.

I've seen a lot of people refusing to edit anything natively since Objective-C land is a "frightening place". That's why using Swift results in a lot more people getting ready to jump into native development and increasing the overall react-native community engagement (see https://github.com/react-native-community/discussions-and-proposals/issues/253)

While it is theoretically possible to use Swift for TurboModule development, it would be a real pain to interop with the JSI layer since that's written in C++. Correct me if I'm wrong, but afaik stuff like calling host functions, interacting with host objects, etc require a separate bridging layer that's built in one of two ways:

Either way you're very limited with C++ language features and can't use C++ types or templates in the second approach since Swift <-> C++ interop isn't fully available yet (see "How developed is C++ interoperability?")

I think that it would make a lot of sense to make use of Swift's Property Wrappers (see Properties, scroll down to "Property Wrappers") for stuff like exporting functions, properties, TurboModules etc, which can then be transformed using codegen internally.

Pseudo code on how this would look for turbo modules:

@ReactModule
class AddTurboModule {

  @ReactMethod
  func add(a: Int, b: Int) -> Int {
    return a + b
  }
}

Pros:

Note that a good developer experience almost always results in a good product.


Then with codegen, swift code per module can be generated to some degree.

Do you have any info to share about that part? @fkgozali

fkgozali commented 3 years ago

@mrousavy

Do you have any info to share about that part?

The long term plan is to allow adding custom generators that consume the schema produced by react-native-codegen. Today, we're just focusing on ObjC and Java generators because that's what React Native core supports directly.

Right now both generators live directly in the package, but later this year, we want to explore providing API to install your own custom generators outside of that package. When then happens, then more generators for other languages can be built and installed as "plugins" to your apps. This allows us to properly support Kotlin, Swift, pure C++, C# at least, like how ObjC and Java are supported.

In summary: we have plans to allow this, but we won't be focusing directly on Swift support in core anytime soon. We see the react-native-codegen project to be the path towards providing more language support in the future.

myckhel commented 3 years ago

Please when will TurboModule be integrated?

I probably taught it would be shipped with React-Native@0.64.

fkgozali commented 3 years ago

Please when will TurboModule be integrated?

All the support is already in Github, and we have enabled it on RNTester app on Android and iOS, so you can take a look at the integration if you like. 0.64 may not have all the pieces in place, but master branch has them. The major thing missing is the integration with 3rd party modules and migration playbook, which we are working on finalizing for later this year.

mrousavy commented 3 years ago

Do TurboModules support multiple callbacks per function? I noticed that with the "legacy" native module system you cannot pass a Callback (RCTResponseSenderBlock) and a Promise (RCTPromiseResolveBlock + RCTPromiseRejectBlock) into the same function, I assume that by building upon JSI this will work out of the box, no?

msageryd commented 3 years ago

Sorry if this is the wrong place. I didn't find any good place to post this possibly turboModules issue.

I just upgraded from RN 0.63.2 to 0.64. And my project doesn't build anymore due to this: shared_timed_mutex' is unavailable: introduced in iOS 10.0 Error located in RCTTurboModuleManager

My build target is ios 10.0. I even tried to bump it to 11. Googling this error brings nothing, which is quite rare when googling RN problems.

Should I post this elsewhere? Any idéas?

Edit:

My bad, sorry. Solved. https://github.com/facebook/react-native/issues/31250

henrymoulton commented 3 years ago

@fkgozali sorry not relevant to this thread but is there documentation on how to disable TurboModules on RNTester? I think it's breaking fast refresh and debugging with Chrome.

fkgozali commented 3 years ago

@henrymoulton

Fast Refresh should work normally, but old Chrome debugger flow won't work because of the lack of sync call support in that env. We're still hashing out the debugger solution later, but a few things you can try:

If you're hitting issue with Fast refresh, make sure you're not connected to the old Chrome debugger flow (the one you launch from dev menu) and see if the problem is solved. If not, would you mind opening a new issue and linking to it here?

To disable TurboModule in RNTester (not recommended but OK for your own exploration), change the flag(s) here:

prakashjais99 commented 3 years ago

I want to use TurboModules to send events from Java to JS using JSI bridge. But i am not able to find a right docs or sample code to proceed. Below is what I have achieved so far.

Code to register and later callback

 void TrimNativeModule::sendKeyEvent(jsi::Runtime &rt, const jsi::Object &arg) {
        if (keyCallback != nullptr) {
            jsi::Function funPtr = keyCallback->getFunction(rt);
            jsInvoker_->invokeAsync([&rt, &funPtr](){
                funPtr.call(rt);
            });
        }
    }

    void TrimNativeModule::registerKeyCallback(jsi::Runtime &rt, const jsi::Function &callback) {
        keyCallback = &callback;
    }

Below is the JS side code

  global.trimModule?.registerKeyCallback(() => {
  console.log("Got Call back");
});

Registration is working but when I tried to do call sendKeyEvet later for an event from Java native function call back is not called back

Any help or pointers on this will be helpful

radelcom commented 3 years ago

@prakashjais99 quick question, I am also in the similar situation but was wondering how where you getting reference to the jsInvoker_ value?

mrousavy commented 3 years ago

@radelcom

iOS

  1. import RCTTurboModule.h
  2. use bridge.jsCallInvoker (RCTBridge)

Android

  1. Use context.getCatalystInstance().getJSCallInvokerHolder() (ReactApplicationContext)
radelcom commented 3 years ago

@mrousavy follow up question... when passing the invoker to the cpp-adapter.cpp file, what is the proper way to cast the invoker?

extern "C" JNIEXPORT void JNICALL Java_com_MyAwesomeModule_initialize(JNIEnv* env, jclass clazz, jlong jsi, jobject invoker) { installMyAwesomeModule(*reinterpret_cast<facebook::jsi::Runtime *>(jsi), invoker); }

my cpp file has this declaration void installMyAwesomeModule(jsi::Runtime& jsiRuntime, std::shared_ptr<react::CallInvoker> jsInvoker)

kidroca commented 3 years ago

Another workaround to "old debug flow not working": If you're using Hermes you can follow the steps here: https://reactnative.dev/docs/hermes#debugging-js-on-hermes-using-google-chromes-devtools This doesn't require Flipper

rangav commented 3 years ago

Here is roadmap update from react native team

https://reactnative.dev/blog/2021/08/19/h2-2021

Looks like they are planning to release new architecture by end of this year.

yaaliuzhipeng commented 2 years ago

@mrousavy follow up question... when passing the invoker to the cpp-adapter.cpp file, what is the proper way to cast the invoker?

extern "C" JNIEXPORT void JNICALL Java_com_MyAwesomeModule_initialize(JNIEnv* env, jclass clazz, jlong jsi, jobject invoker) { installMyAwesomeModule(*reinterpret_cast<facebook::jsi::Runtime *>(jsi), invoker); }

my cpp file has this declaration void installMyAwesomeModule(jsi::Runtime& jsiRuntime, std::shared_ptr<react::CallInvoker> jsInvoker)

dude , did you solved this cast problem ? I encountered this too .

mrousavy commented 2 years ago

Here's the magic 🪄:

// from java:
CallInvokerHolderImpl callInvoker = (CallInvokerHolderImpl) context.getCatalystInstance().getJSCallInvokerHolder()
initialize(runtime.get(), callInvoker)
// in c++:
extern "C" JNIEXPORT void JNICALL Java_com_mrousavy_MyModule_initialize(JNIEnv* env, jclass clazz, jlong jsiRuntime, jni::alias_ref<facebook::react::CallInvokerHolder::javaobject> callInvoker);

// cast:
auto jsCallInvoker = jsCallInvokerHolder->cthis()->getCallInvoker();

But keep in mind that this is pretty unrelated to the issue.

yaaliuzhipeng commented 2 years ago

hey ! guys ! As the rn0.68.0 released the new arch feature; I'm about digging into rn's new archi adoption; 👉🏻 https://reactnative.dev/docs/next/new-architecture-library-android I met this gradle error when building

Could not find method react() for arguments [build_cbn8oel6m6i7bg72w8bsdvl$_run_closure3$_closure11@ef70746] on object of type org.gradle.api.internal.artifacts.dsl.dependencies.DefaultDependencyHandler

my library 'module-level' react spec looks like below

react {
    libraryName = "Test"
    codegenJavaPackageName = "com.iturbomodule.test"
    root = rootProject.file("..")
    jsRootDir = rootProject.file("../js/")
    reactNativeDir = rootProject.file("../react-native/")
    codegenDir = rootProject.file("../react-native-codegen/")
}
cortinico commented 2 years ago

@yaaliuzhipeng As a rule of thumb, report issues like this one either on github.com/reactwg/react-native-new-architecture/ (if you don't have access yet, you can apply for it with the form in the README).

Could not find method react() for arguments

This happens as you don't have the React Gradle Plugin applied in that module. I would say that you miss a:

plugins {
  id("com.facebook.react")
}

but not knowing your full gradle file setup, makes hard to say if this is the only issue

yaaliuzhipeng commented 2 years ago

@yaaliuzhipeng As a rule of thumb, report issues like this one either on github.com/reactwg/react-native-new-architecture/ (if you don't have access yet, you can apply for it with the form in the README).

Could not find method react() for arguments

This happens as you don't have the React Gradle Plugin applied in that module. I would say that you miss a:

plugins {
  id("com.facebook.react")
}

but not knowing your full gradle file setup, makes hard to say if this is the only issue

love you dude, I did missed the plugin, everything goes fine after this plugin's added ! ! ! guess I have to learn gradle for a while 😅