[HOLD for payment 2024-04-25] [Tracking] [Performance] Enable the new Fabric architecture

mvtglobally commented 2 years ago

GitHub Project: https://github.com/orgs/Expensify/projects/35

Main New Arch PR: https://github.com/Expensify/App/pull/13767

Context

New render engine: https://reactnative.dev/architecture/fabric-renderer Summary of findings 1) There are 4 main components of RN: React, JavaScript, a series of elements collectively known as the “bridge”, and the native elements/modules. Each of these components are getting complimentary upgrades. 2) The notable React upgrades discussed are concurrent rendering, synchronous event callbacks, and priority rendering queues. Together those improvements basically allow for the UI thread to perform heavy rendering work in the background without compromising quick response times to user events such as scrolling and navigation gestures. 3) In the existing RN architecture, the JS and native sides are essentially unaware of each other. All communications are serialized JSON messages sent asynchronously over the bridge. 4) When JavaScript communicates with a web browser, the browser objects hold direct references to native input elements which are controlled internally by the browser. So an HTML checkbox in a browser actually internally holds a reference to a native macOS or Windows checkbox that’s managed by the browser. Similarly, JSI (JavaScript Interface) is a new piece of RN that enables direct communication between JS and the native side, because JSI internally manages direct references to C++ host objects (similar to the HTMLInput element on a browser). This means that messages sent between JS and native modules no longer need to be JSON-serialized or sent over the single bottleneck communication channel. It also means messages can be sent either synchronously or asynchronously. 6) At a high level, the traditional RN bridge manages two tasks: computing layout via a component tree called the “shadow tree” (that shadows the React tree), and managing native modules such as the camera. In the new architecture, these are broken into two distinct pieces which facebook is calling Fabric and TurboModules, respectively. 6) Fabric, the new UI manager, leverages JSI to create the shadow tree in C++ directly, cutting out the middle man and dramatically improving performance. It also exposes UI operations on both sides, so the native side and JS side can both directly manipulate the shadow tree without having to send serialized messages back and forth across the bridge. 7) In the existing native module system, all the native modules need to be loaded when the app launches so that they’re ready and waiting for commands to be sent over the bridge. Turbo Modules have direct references to native modules via JSI, which gives two main benefits. First, native modules can be lazy-loaded (only when really needed), which should dramatically improve app startup time, especially on apps that use lots of different native modules. Second, once loaded the modules will be more performant because communications no longer need to be JSON-serialized and sent across the bridge. 8) Lastly, there is a tool I don’t fully understand called CodeGen. It leverages TypeScript to generate the interface files needed by Fabric and TurboModules. This compile-time type safety means that the code counterparts from both the native and JS worlds can trust eachother without any runtime checks, which means smaller bundle sizes and faster execution.

Requirements

Migrate off setNativeProps

Migrate off findNodeHandle

[x] https://github.com/Expensify/App/issues/11635

Bidirectional scrolling

[x] testing and making sure the bidirectional pagination features we’ve built work with Fabric https://github.com/Expensify/react-native/pull/32

Dependency upgrades

[x] Upgrade to Reanimated 3 https://github.com/Expensify/App/pull/11567
[x] react-native-svg https://github.com/Expensify/App/pull/12140
[x] Upgrade react-native https://github.com/Expensify/App/issues/12177
[x] React Native Picker
- Has PR: https://github.com/react-native-picker/picker/pull/456
[x] progress-bar-android
- Has PR: https://github.com/react-native-progress-view/progress-bar-android/pull/90/
[x] React Native PDF
- Has PR: https://github.com/wonday/react-native-pdf/pull/700
[x] UrbanAirship
[ ] ~~FastImage~~ no longer in use
- Has PR: https://github.com/DylanVann/react-native-fast-image/pull/951

Known bugs

[ ] Bold fonts don't work with Fabric enabled
[x] https://github.com/Expensify/App/issues/14365
[x] https://github.com/Expensify/App/issues/14366
[x] https://github.com/Expensify/App/issues/14367
[x] https://github.com/Expensify/App/issues/14368
[x] https://github.com/Expensify/App/issues/14369
[x] https://github.com/Expensify/App/issues/14370
[x] https://github.com/Expensify/App/issues/14371
[x] https://github.com/Expensify/App/issues/14372
[x] https://github.com/Expensify/App/issues/16795
[x] https://github.com/Expensify/App/issues/16796
[x] https://github.com/Expensify/App/issues/16799
[x] https://github.com/Expensify/App/issues/16806
[x] https://github.com/Expensify/App/issues/16807
[x] https://github.com/Expensify/App/issues/16808
[x] https://github.com/Expensify/App/issues/16809
[x] https://github.com/Expensify/App/issues/16810
[x] https://github.com/Expensify/App/issues/16811
[x] https://github.com/Expensify/App/issues/16814
[x] https://github.com/Expensify/App/issues/16815
[x] https://github.com/Expensify/App/issues/16824

Expensify/Expensify Issue URL: Issue reported by: @roryabraham Slack conversation: https://expensify.slack.com/archives/C01GTK53T8Q/p1648743262264529

roryabraham commented 2 years ago

I'm going to create a separate tracking issue to enable React 18 on all platforms. Completing this issue and turning on Fabric will be one of those requirements.

Edit: new issue here: https://github.com/Expensify/App/issues/11680, removed the RNW concerns from this issue

stitesExpensify commented 2 years ago

Almost done migrating off of setNativeProps. Other requirements are also moving along!

rushatgabhane commented 2 years ago

@stitesExpensify we can edit the OP and check off ✅ the migration from setNativeProps

The final PR has been on production since 4 days https://github.com/Expensify/App/pull/11673#issuecomment-1278213202

stitesExpensify commented 2 years ago

Hm it looks like this one isn't closed? https://github.com/Expensify/App/issues/11053

luacmartins commented 2 years ago

https://github.com/Expensify/App/pull/11673 was merged, but the issue is still open for some reason. I think we can just close it.

stitesExpensify commented 2 years ago

Cool! Thanks guys!

roryabraham commented 2 years ago

The last two items are moving along nicely

@janicduplessis is making progress on upgrading bidirectional scrolling to work with Fabric and the PR is in review.
it sounds like @hannojg figured out how to finish the Reanimated 3 upgrade without us waiting for the official release. So that should be almost ready to move forward as well

Then comes the big step – actually make the switch to Fabric and test like we've never tested before to make sure we don't break stuff. I'd recommend we get like 3 C+ reviewers on that PR to make sure we have lots of eyes on it.

trjExpensify commented 2 years ago

Niceee! 🚀 Let's link those in the OP - I've added them both.

I'd recommend we get like 3 C+ reviewers on that PR to make sure we have lots of eyes on it.

Yeah, I agree with this sentiment.

JmillsExpensify commented 2 years ago

Totally, the more C+ the merrier to make sure this is smooth.

roryabraham commented 2 years ago

testing and making sure the bidirectional pagination features we’ve built work with Fabric

This was done in this PR and our bidirectional pagination features have been upgraded to work with Fabric. All that's left to check this item off is upgrading E/App to version 0.70.5 of our fork.

Edit: PR to upgrade E/App to 0.70.5 is ready for review: https://github.com/Expensify/App/pull/11975

hannojg commented 2 years ago

Fixed the tests on the REA 3 PR, should be good now: https://github.com/Expensify/App/pull/11567

JmillsExpensify commented 2 years ago

Still working through REA 3 PR.

JmillsExpensify commented 1 year ago

Expecting to merge the PR today.

roryabraham commented 1 year ago

REA 3 upgrade was merged yesterday 🎉

But we discovered another blocker – react-native-svg does not yet support TurboModules, WIP implementation here.

Edit: Actually it looks like react-native-svg has Fabric support as of v13, so we just need to upgrade to the latest.

AndrewGable commented 1 year ago

I also was not able to get Android to build on Fabric, @janicduplessis said he might be able to take a look.

https://expensify.slack.com/archives/C01GTK53T8Q/p1666738055116419?thread_ts=1666696535.015619&cid=C01GTK53T8Q

AndrewGable commented 1 year ago

On iOS with Fabric + Turbo Modules I am experiencing an issue while navigating backwards from a chat to the Left Hand Nav. Then once I try to click back into the chat, the Left Hand Nav "flashes" as well.

https://user-images.githubusercontent.com/2838819/198100114-e5be5491-ecac-44e2-95ec-874e6a841c0b.mov

AndrewGable commented 1 year ago

@janicduplessis Fixed the Android build issues, I opened a WIP PR linked above with those fixes included. The "flickering" issue occurs on both Android & iOS now.

roryabraham commented 1 year ago

REA 3 upgrade is complete

roryabraham commented 1 year ago

Looks like we need to revert https://github.com/Expensify/App/pull/12140 because it's causing a crash. So we'll need to work that crash out and merge it back in order to get Fabric enabled

roryabraham commented 1 year ago

According to this thread, it sounds like we need to upgrade our react-native fork, created an issue here and added to OP

roryabraham commented 1 year ago

We should investigate if react-native-pdf needs to be upgraded too

JmillsExpensify commented 1 year ago

@roryabraham What's blocking us at this point now that https://github.com/Expensify/App/issues/12177 is merged?

rushatgabhane commented 1 year ago

I think it's PR https://github.com/Expensify/App/pull/12167. We could assign multiple reviewers to it and try to break everything and anything

JmillsExpensify commented 1 year ago

Removing this from WAQ in light of this discussion. We'll continue to work on this outside of the immediate product roadmap though, for clarity.

roryabraham commented 1 year ago

Posted here to propose handing this issue off

j-piasecki commented 1 year ago

Hi 👋, I'm Jakub from SWM. Together with @WoLewicki, we'll be working on enabling the new architecture.

WoLewicki commented 1 year ago

Hi, I'm Wojciech from SWM, and I'll be working with @j-piasecki 😅

AndrewGable commented 1 year ago

Welcome welcome! Excited to see the results.

roryabraham commented 1 year ago

Cross-posting updates from slack – sounds like the main thing happening this week is working on a fabric implementation in react-native-picker/picker

roryabraham commented 1 year ago

Regular updates coming in on slack, but it sounds like the main things happening right now are:

Migrating picker
Migrating progress-bar-android

roryabraham commented 1 year ago

Going to unassign @stitesExpensify and myself from this, because this is being handled by @j-piasecki and @WoLewicki going forward.

WoLewicki commented 1 year ago

Libraries with native components to migrate (with corresponding PRs if present already):

roryabraham commented 1 year ago

Asked the folks at UrbanAirship if they have any plans of upgrading https://github.com/urbanairship/react-native-module any time soon via our shared slack channel here

melvin-bot[bot] commented 1 year ago

This issue has not been updated in over 15 days. @j-piasecki, @WoLewicki eroding to Monthly issue.

P.S. Is everyone reading this sure this is really a near-term priority? Be brave: if you disagree, go ahead and close it out. If someone disagrees, they'll reopen it, and if they don't: one less thing to do!

dylanexpensify commented 1 year ago

Coming from this convo, added issues to the Known Bug section

roryabraham commented 1 year ago

Going to re-assign myself to this issue so that I can work on upgrading urbanairship-react-native to the new @ua/react-native-airship package, on the old architecture to start.

Julesssss commented 1 year ago

Any updates here, or is there another issue to be following?

roryabraham commented 1 year ago

There have been many incremental updates in the #expensify-swm channel. I think there are only a few other libraries left to migrate to TurboModules – react-native-firebase/(analytics/perf/crashlytics), then every library we currently use will be Fabric and TurboModules-enabled.

I just kicked off a new test build and then we'll have Applause run another full regression suite against the Fabric-enabled app.

roryabraham commented 1 year ago

Starting a new round of Fabric testing: https://expensify.slack.com/archives/C9YU7BX5M/p1680021248483969

roryabraham commented 1 year ago

I believe Applause will be posting their summary of the second round of regression tests in a few minutes. When they do, @j-piasecki @WoLewicki if you could please comment on each issue so that I can assign them to you, that would be great. Thanks! (divide them up however you want)

Also, I'm thinking that maybe we should start using a GitHub Project to track this project?

mvtglobally commented 1 year ago

Fabric testing is completed

QA identified multiple issues

Android Deploy blocker - https://github.com/Expensify/App/issues/16692 Since user is not able to login, we were not able to validate additional functionality

iOS

WoLewicki commented 1 year ago

@roryabraham we assigned ourselves to all issues. As for the Github Project, yeah, why not? 😅

roryabraham commented 1 year ago

Created a GitHub project to track this: https://github.com/orgs/Expensify/projects/35/views/1

I also created an internal ticket to make the project public

roryabraham commented 1 year ago

Requested another round of testing on the Fabric branch here

roryabraham commented 1 year ago

@j-piasecki @WoLewicki I think this issue has been going on long enough that we should schedule a call to chat about it and get aligned on where we're at w/ it and how best to move forward.

It seems like at this point we've logged a ton of issues and hopefully fixed many of them, so the next step now should be:

Make sure the fabric branch is up-to-date and has all conflicts resolved.
Manually retest all the reported issues tracked in the GitHub Project, make sure they've been addressed
Run a new test build
Have Applause do another round of QA on the Fabric-enabled app
Hopefully merge the PR

j-piasecki commented 1 year ago

@roryabraham What do you think about scheduling the call sometime next week? We have an idea about what is causing issues with react-freeze and hopefully we will be able to fix them by that time as freeze is much more important after the navigation refactor. In the meantime, we will start retesting all the issues so we have the results ready for the call.

melvin-bot[bot] commented 1 year ago

Looks like something related to react-navigation may have been mentioned in this issue discussion.

As a reminder, please make sure that all proposals are not workarounds and that any and all attempt to fix the issue holistically have been made before proceeding with a solution. Proposals to change our DeprecatedCustomActions.js files should not be accepted.

Feel free to drop a note in #expensify-open-source with any questions.

roryabraham commented 1 year ago

Created a LettuceMeet poll to schedule a sync for the second half of next week.

roryabraham commented 1 year ago

Copying the summary from our call:

Currently only two of the known issues are reproducible and they're minor
There is a tricky bug w/ Freeze, stemming from the way Reanimated interacts with the shadow tree. As I understand it, @tomekzaw has a solution in mind. Maybe he can provide more context there.
- Edit: This should be fixed in Reanimated 3.4.0, so we'll upgrade to that before putting the Fabric PR up for review
We want to upgrade to 0.72 before putting the Fabric upgrade up for review because it includes bug fixes, config changes, and new architecture performance improvements
The last thing we'll want to do before putting the New Arch PR up for review is a quick audit of all the patches that we have there to remove those that aren't needed, make sure we have upstream PRs for those that are, and to list the upstream PR for each patch in the PR.
We feel that the branch is stable enough and the bugs are minor enough that we aren't going to do another round of Applause QA in a test build. Instead, we'll focus on getting this actually merged, (hopefully within the next few weeks? :fingers_crossed:)

tomekzaw commented 1 year ago

There is a tricky bug w/ Freeze, stemming from the way Reanimated interacts with the shadow tree. As I understand it, @tomekzaw has a solution in mind. Maybe he can provide more context there.

@j-piasecki investigated the regression in the app and managed to prepare a minimal repro of the underlying issue with Reanimated 3.1.0. The problem seems to be gone if we use the new algorithm for updating props on Fabric (https://github.com/software-mansion/react-native-reanimated/pull/4075) which has just been merged and will be released in 3.4.0.

Expensify / App