Support third-party 64-bit libraries on Android

corbt commented 9 years ago

Excerpt from the Android Developers Blog article, Get your apps ready for the 64-bit requirement: [added by @hramos]

The 64-bit requirement: what it means for developers

Starting August 1, 2019:

All new apps and app updates that include native code are required to provide 64-bit versions in addition to 32-bit versions when publishing to Google Play.
Extension: Google Play will continue to accept 32-bit only updates to existing games that use Unity 5.6 or older until August 2021.

Starting August 1, 2021:

Google Play will stop serving apps without 64-bit versions on 64-bit capable devices, meaning they will no longer be available in the Play Store on those devices.
This will include games built with Unity 5.6 or older.

The requirement does not apply to:

APKs or app bundles explicitly targeting Wear OS or Android TV, which are form factors that do not currently support 64-bit code.
APKs or app bundles that are not distributed to devices running Android 9 Pie or later.

Original Issue

Hey, it appears that React Native on Android doesn't provide a 64-bit version of the libreactnativejni.so native library, which can cause compatibility issues on 64-bit devices. I ran into this while attempting to integrate React Native with a large existing application I'm developing.

Steps to reproduce:

Create a new application with react-native init.
In the "android/app/build.gradle" file, add compile 'io.realm:realm-android:0.82.2' to the end of the dependencies {} block. (Note that I don't think this is an issue specifically caused by Realm, it's just an example dependency with a 64-bit native library).
Attempt to run on a Nexus 9.

Stack trace:

E/AndroidRuntime(32171): FATAL EXCEPTION: main
E/AndroidRuntime(32171): Process: com.androidemberall, PID: 32171
E/AndroidRuntime(32171): java.lang.UnsatisfiedLinkError: could find DSO to load: libreactnativejni.so
E/AndroidRuntime(32171):    at com.facebook.soloader.SoLoader.loadLibraryBySoName(SoLoader.java:213)
E/AndroidRuntime(32171):    at com.facebook.soloader.SoLoader.loadLibrary(SoLoader.java:178)
E/AndroidRuntime(32171):    at com.facebook.react.bridge.JSCJavaScriptExecutor.<clinit>(JSCJavaScriptExecutor.java:19)
...

I believe this to be caused by the shared library loading logic explained in this answer on StackOverflow. Since Realm includes a 64-bit native binary, the system no longer automatically falls back to the 32-bit version of libreactnativejni.so provided by React.

This bug is blocking me from integrating React Native into my existing application, which I'm really excited to try!

mkonicek commented 9 years ago

Thanks for reporting! Yes we don't provide 64-bit version of the native code and the system should always fall back to 32-bit. I believe @kmagiera has context on why we don't provide 64-bit version.

corbt commented 9 years ago

Like the stackoverflow answer says, the fallback doesn't work if there are other 64-bit native libraries. Most Android projects use a number of 3rd-party libraries, and any that include native 64-bit code will cause React Native to fail.

I think that Android's policy on this is strange (the way one library is loaded shouldn't affect another that doesn't depend on it), but as you can see if you follow my steps to reproduce, it does in fact fail. As more libraries switch to native 64-bit support and React Native gets used in more heterogeneous projects this will become more of an issue.

kmagiera commented 9 years ago

It definitely matters as all native libs share address space.

Workaround would be to follow build.grade from our examples and add the following block to defaultConfig:

ndk {
  abiFilters "armeabi-v7a", "x86"
}

see: https://github.com/facebook/react-native/blob/master/Examples/SampleApp/android/app/build.gradle

corbt commented 9 years ago

Yeah, I tested with that ndk block as well (it actually is included in the default project config generated by react-native init) and the issue still stands.

corbt commented 9 years ago

I've uploaded an example failing project here, in case it helps with debugging.

The only changes to the default React-Native-generated project are the addition of the README and the addition of the realm-android dependency.

corbt commented 9 years ago

Furthermore, I can confirm that installing the apk in 32-bit mode works on the nexus 9, as found in this SO answer (adb install --abi armeabi-v7a app-debug.apk)

corbt commented 9 years ago

I figured out a workaround in the short term (disabling all ARM64 binaries) and wrote it up in a blog. However, the right solution definitely needs to be proper ARM64 support, because that's the way the device ecosystem is moving).

If it isn't included because of concerns over binary size, maybe switching to the packaged V8 interpreter would help? Any ARM64 devices should come with it included, and perhaps falling back to the vendored JSC for devices where V8 isn't available could work.

ide commented 9 years ago

The fragmentation of running on different JS VMs is worth avoiding. It already kind of sucks that iOS 7/8/9 support different features, and the JS VM should be only a couple MB anyway.

Can ABI splits be used to address concerns about APK size? The idea is that you generate three APKs for 32bit, 64bit, simulator, and distribute them separately.

corbt commented 9 years ago

@ide yes, splits can be used to keep size down, although Google doesn't recommend it: "Although we encourage you to develop and publish a single APK that supports as many device configurations as possible, doing so is sometimes not possible" (emphasis theirs).

I think a reasonable solution would be to generate a monolithic APK by default with x86/arm/arm64 support, and allow individual developers to go through the hassle of producing multiple APKs if their app requires it. I don't think that just ignoring ARM64 will be a good long-term compromise -- it will become harder and harder to use 3rd-party libraries with React Native as more include ARM64 support and more 64-bit devices come to market.

ide commented 9 years ago

I kind of understand Google's solution though that's how Apple operated for almost a decade and they're moving to architecture-/device-specific binaries (but doing it all for you with their bitcode compiler).

allow individual developers to go through the hassle of producing multiple APKs if their app requires it

I like this solution as a starting point.

Things to track for 64-bit support are:

[x] 64-bit JSC: https://github.com/facebook/android-jsc/issues/5
[ ] Targeting armv8 with Buck: https://github.com/facebook/buck/issues/414
[ ] Building all the dependencies with 64-bit support (I tried this and some just don't work)

All of these are up-for-grabs. @kmagiera can you tell us if 64-bit is something that Facebook is actively interested in, or should people just start working on these items?

kmagiera commented 9 years ago

@ide I don't think we have anyone focusing on this at the moment. What are the specific things that doesn't compile for armv8? Except from JSC we build all the first-party and third-party libs RN depends on with BUCK for arm64 so it shouldn't take much to make android makefiles build them as well.

Speaking of split builds I don't think Google's recommendation apply in the case when 80% of the APK size comes from native libs. Don't have exact values with me at the moment but from what I remember JSC lib is around 2.3M for armv7 and 2.5M for x86 (uncompressed) which adds around 1.3M to the APK size. Considering the fact that arm64 are quite rare and all of them can run armv7 binaries (JSC with JIT enabled doesn't work with libhoudini if I recall correctly from our experiments, so I consider x86 android devices incapable of running armv7 in this case) I don't think we should default to armv7+arm64+x86 builds. My suggestion would be still to default to x86+armv7 but provide config for split builds for arm64 once it's supported

ide commented 9 years ago

@kmagiera - OK. So if people are interested in 64-bit support they should contribute.

I hit an issue with Google's double-conversion library but I could have specified the wrong compiler options. I didn't spend much time looking into it.

kmagiera commented 9 years ago

@ide feel free to send PR, I'll update here once we start working on this but I don't expect this to happen within the next couple of weeks.

LegNeato commented 9 years ago

FWIW Facebook does ABI apk splitting (and others). Whenever we ship we ship ~14 apks to the store. Out of that 14 we had a fallback that supported everything (no splitting) but virtually no device ever installed it because the others were sufficient.

Matthew-C commented 9 years ago

Hi, any plan to solve this issue in the near future? It simply blocks us using react-native on Nexus 9.

ide commented 9 years ago

@Matthew-C I haven't heard any updates. If you need this feature, please help out!

mkonicek commented 9 years ago

@Matthew-C This workaround might unblock you (see the discussion above): https://corbt.com/posts/2015/09/18/mixing-32-and-64bit-dependencies-in-android.html

atticoos commented 9 years ago

+1 on this, our product has some 64bit dependencies as well

lklots commented 8 years ago

@corbt thank you for publishing the workaround, works fine! https://corbt.com/posts/2015/09/18/mixing-32-and-64bit-dependencies-in-android.html

mkonicek commented 8 years ago

Hi there! This issue is being closed because it has been inactive for a while.

But don't worry, it will live on with ProductPains! Check out its new home: https://productpains.com/post/react-native/android-react-native-for-android-is-incompatible-with-3rd-party-64-bit-libraries

ProductPains helps the community prioritize the most important issues thanks to its voting feature. It is easy to use - just login with GitHub.

Also, if this issue is a bug, please consider sending a PR with a fix. We're a small team and rely on the community for bug fixes of issues that don't affect fb apps.

MattFoley commented 8 years ago

@ide and @mkonicek, I think we should consider re-opening this. Am I understanding correctly that the underlying issue is that JavaScriptCore hasn't been compiled to 64bit for Android?

Is the Android architecture landscape wildly different from the iOS architecture landscape? On iOS, you've been required to support arm64 over a year. If Google takes on this stance, than this is no longer a product pain, and becomes a massive blocker for many developers using React Native.

I argue that due to this, as well as the large performance gain 64bit architectures have over 32bit, this should be reopened and prioritized. If you can provide direction on what needs to be done, it's possible that we can devote time to helping with this effort, as this will quickly become a blocker for us here at Skillz (game developers need all the performance they can get).

Thanks!

csotiriou commented 8 years ago

This is indeed a blocker for us, too. We are really interested into using react native in our company, but the lack of 64-bit architectures on android is a real show-stopper, since we definitely need to use some C++ dependencies as well (we build some augmented reality apps)

Throwing away all 64 bit libs as indicated in this post ( https://corbt.com/posts/2015/09/18/mixing-32-and-64bit-dependencies-in-android.html ) is not an actual workaround, it's a hack, that can only get you this far.

Can we at least have an indication regarding this issue's progress (if any)?

MattFoley commented 8 years ago

@csotiriou from what I can tell, the best description of the work needed done is expanded upon here by @ide. Item number three seems to be a pretty big rabbit hole at the moment, with no real clear definition. The first two items however are accurate. Item one seems to be a sizable chunk of work, and less defined, whereas item two seems to be smaller and has what seems to be a clear path forward laid out by this issue on Buck.

ide commented 8 years ago

A good starting point would be to address all the checkboxes in my post above. Get all dependencies working with 64-bit, and add support for armv8 to Buck. Once that's done, it makes more sense to revive the conversation about getting RN compiling for 64-bit.

LegNeato commented 8 years ago

FWIW I added an untested patch to https://github.com/facebook/buck/issues/414

felipecsl commented 8 years ago

We found a slightly better workaround for this issue than the one mentioned by @kmagiera, based on this blog post We're doing something like this:

ext.nativeLibsExcludeList = Arrays.asList('libFoo.so', 'libBar.so', ...)
packagingOptions { 
  exclude {
    // ... other excluded files
    for (String excludedLib : nativeLibsExcludeList) {
        exclude "lib/mips64/$excludedLib"
        exclude "lib/arm64-v8a/$excludedLib"
        exclude "lib/x86_64/$excludedLib"
    }
  }
}

nativeLibsExcludeList has to include all the native libraries included in your 64bit platform directories. You can find that out by opening your APK and looking into which files are there. This is better than the abiFilters because that one may prevent you from publishing your app update if you have previou(s) APK(s) that include explicit support for one or more of the 64bit native platforms. If that happens, Google Play Dashboard won't let you publish the update with an error because you can't downgrade the existing apps or something similar. With this exclude solution, you're technically still supporting all the possible native platforms and Android will automatically serve the 32bit libraries for 64bit devices for you.

lorrainefox commented 8 years ago

We are well into an important mobile app that is ReactNative for Android/iOS and can resort to the removal of the 64 bit libs, but a solution is very important to us. Adding this comment to up vote work on the solution.

leeight commented 8 years ago

+1

LegNeato commented 8 years ago

[x] 64-bit JSC: facebook/android-jsc#5 -- @leeight has a PR up 🎉
[x] Targeting armv8 with Buck: facebook/buck#414 -- I fixed this in Buck master 🍻
[ ] Building all the dependencies with 64-bit support -- No one working on it 😢

There should be enough working here with @leeight's outstanding PR and my landed Buck change to start working on the react-native 64bit dependencies. Any takers?

artdent commented 7 years ago

Here's a slightly more robust variant of the packagingOptions workaround mentioned above:

    packagingOptions {
        exclude '/lib/mips64/**'
        exclude '/lib/arm64-v8a/**'
        exclude '/lib/x86_64/**'
    }

This excludes all 64-bit libraries from the apk, without requiring you to enumerate them specifically.

Now, is this better than the other workaround of specifically targeting just the armeabi-v7a and x86 architectures in the ndk block? I don't know. I do know that the device targeting workaround caused the Play Store to tell me that my app would target 40 fewer device models, out of about 9500. But I couldn't figure out how to get the list of what those devices are, so I don't know if any are actually used in the real world.

antoinerousseau commented 7 years ago

I implemented @artdent's solution but I still get this from a Nexus 5...

java.lang.UnsatisfiedLinkError: couldn't find DSO to load: libreactnativejni.so

wqqgt commented 7 years ago

I have the same problem !

aem commented 7 years ago

@corbt's solution worked to get our app running, but ideally we'd like to be running in 64-bit mode. Is getting the binaries built to target 64-bit at all on Facebook's radar?

lorrainefox commented 7 years ago

I have the same request, that we can run in 64-bit mode.

On Wed, Feb 8, 2017 at 10:04 AM, Adam Markon notifications@github.com wrote:

@corbt https://github.com/corbt's solution worked to get our app running, but ideally we'd like to be running in 64-bit mode. Is getting the binaries built to target 64-bit at all on Facebook's radar?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/facebook/react-native/issues/2814#issuecomment-278411247, or mute the thread https://github.com/notifications/unsubscribe-auth/ABHA4TKlVS6-Fwy_qGGFHJ17fuAaK7peks5ragOUgaJpZM4F_XHG .

denisk20 commented 7 years ago

I have tried the workarounds above, one which excludes 64bit binaries and another which targets armeabi-v7a and x86 architectures, and in both cases I'm getting the same error when trying on x86_64 emulator (API 25):

com.facebook.react.bridge.UnexpectedNativeTypeException: TypeError: expected dynamic type `int64', but had type `null'
  at com.facebook.react.bridge.ReadableNativeMap.getInt(Native Method)
  at com.facebook.react.devsupport.StackTraceHelper.convertJsStackTrace(StackTraceHelper.java:103)
  at com.facebook.react.devsupport.DevSupportManagerImpl$3.run(DevSupportManagerImpl.java:287)
  at android.os.Handler.handleCallback(Handler.java:751)
  at android.os.Handler.dispatchMessage(Handler.java:95)
  at android.os.Looper.loop(Looper.java:154)
  at android.app.ActivityThread.main(ActivityThread.java:6119)
  at java.lang.reflect.Method.invoke(Native Method)
  at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:886)
  at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:776)

What might be causing it?

petterh commented 7 years ago

@denisk20: Your UnexpectedNativeTypeException is almost certainly caused by a missing lineNumber in a JS stack trace. I created a PR to fix it – I gather my fix didn't make it into 0.43, but it is in the master branch, so might be included in the next release. (?)

EDIT

There's an issue for this, though it's not related to the 32/64-bit issue, I think.

anudeepreddygopu commented 7 years ago

Hey, can anyone help with the question I have posted here http://stackoverflow.com/questions/43368926/android-native-script-java-lang-unsatisfiedlinkerrorcould-find-dso-to-load-lib

csotiriou commented 7 years ago

@LegNeato @leeight regarding the pull request for jxcore https://github.com/facebook/android-jsc/pull/19

Think that if this gets merged, and react native will depend on it for 64 bit support, RN will be forced to drop support for devices it currently supports. Something that needs to be heavily considered. I for one like to look fordward, so I believe that support for Android 4.1 is not needed anymore.

Regarding this issue, if this pull request works then I believe we need to proceed with fixing this issue after it gets merged, as the inability to use 64 bit libraries is slowly killing adoption of RN. It is also something that kills projects as it is not mentioned beforehand in the main documentation, and many projects start without knowing this.

LegNeato commented 7 years ago

I don't think merginf will drop support for any phones...worst-case you can still only build the 32bit flavor of this lib. Also, the API version bump is just for building this, not for running AFAIK.

I could be wrong though, been a while since I looked at the specifics.

vipcxj commented 7 years ago

new jsc this project seems provide a improved jsc which support 64bit device. I haven't try it.

julee commented 7 years ago

We tested, if a machine support arm64-v8a, compared to armeabi-v7a, arm64-v8a has more performance advantages. Especially if we need CPU to do multimedia processing (e.g decoding, encoding), some VR operation based on CPU calculation, the performance of arm64-v8a is twice than armeabi-v7a.

If an app wants to use or develop the related functions described above, React Native is completely unavailable because Reactive Native does not support arm64. Since an app entire process can only run under an abi, this not available means that the entire app is completely can't use React Native, not only a component.

It is difficult to imagine almost all new phones are arm64 today, Android app can only run on the 32-bit architecture. In contrast, on the iPhone, Apple has even no longer support 32-bit app.

Bye, Native React, Bye, All These Outdated technology.

teaserbot commented 7 years ago

@julee so what you using now? much better stuff, me thinks :)

stale[bot] commented 6 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Maybe the issue has been fixed in a recent release, or perhaps it is not affecting a lot of people. If you think this issue should definitely remain open, please let us know why. Thank you for your contributions.

simonracz commented 6 years ago

Do not close. This issue is important.

trevorah commented 6 years ago

⚠️ via https://android-developers.googleblog.com/2017/12/improving-app-security-and-performance.html

In August 2019, Play will require that new apps and app updates with native libraries provide 64-bit versions in addition to their 32-bit versions.

vipcxj commented 6 years ago

The dead line is coming~

SandroMachado commented 6 years ago

I raised this concern on reddit and @grabbou or someone with his nickname :) replied.

I wouldn't worry too much about the fact that there was no update or clarification yet.

Facebook and many other companies, including Microsoft and their Skype app, are extensively using React Native in production. Their employees are already contributing heavily to the repository on regular basis, working on whatever is on their internal roadmap for the framework.

The support will land, it's just a matter of time. I will highlight this concern at the closest meeting and get attention from Facebook.

So, I think that this will not be a problem in the near future.

grabbou commented 6 years ago

Yeah, it was me xd

Just to clarify, by saying "will land", I expressed my belief (which seems to be pretty reasonable as I can't imagine major app being taken down because of this).

jcampbell05 commented 6 years ago

Will be big help if this could land soon. It would be nice if this could form some kind of roadmap as currently React Native seems to lack one.

ide commented 6 years ago

There is nothing to land soon (there are no PRs or work in progress). There is the August 2019 deadline and enough apps rely on RN that I'm relatively confident 64-bit support will be ready by then (~18 months out).

facebook / react-native