rhomobile / rhodes

The Rhodes framework is a platform for building locally executing, device-optimized mobile applications for all major smartphone devices.
http://tau-platform.com/
MIT License
1.05k stars 237 forks source link

Android Results after a week using aarch64 version #1014

Closed leonardofalk closed 4 years ago

leonardofalk commented 5 years ago

We released x64 builds for our users and then a lot of crashes began to happen, there are many scenarios and we couldn't figure out whats going on and if we could do anything to fix, so we rolled back to Rhodes 6 and had to remove app compatibility with Android 9 temporarily. Also there is data loss when rolling back, because sqlite is 3.x in rhodes 7, while it was 2.x back then.

Some of the errors include crashing the app when performing http operations, crashing after a while without doing anything, crashing upon app startup, and sometimes closing the app suddenly because "Application is throwing too many failures".

Some of the ANR or Failures reported in Android Vitals:

"main" prio=5 tid=1 Native
  | group="main" sCount=1 dsCount=0 flags=1 obj=0x758334d0 self=0xe64a7000
  | sysTid=1959 nice=0 cgrp=bg_cached sched=0/0 handle=0xeaf084a8
  | state=S schedstat=( 3246444834 7145700127 9227 ) utm=215 stm=109 core=2 HZ=100
  | stack=0xff2cd000-0xff2cf000 stackSize=8MB
  | held mutexes=
  #00  pc 0000000000054f28  /system/lib/libc.so (__pselect6+20)
  #01  pc 0000000000023c4f  /system/lib/libc.so (select+58)
  #02  pc 00000000005fdd54  /data/app/com.app.name-ctg2CAxvljcdQISnnN85ug==/lib/arm/librhodes.so (rho::net::CURLNetRequest::CURLHolder::perform()+412)
  at com.rhomobile.rhodes.RhodesService.callActivationCallback (Native method)
  at com.rhomobile.rhodes.RhodesService.handleAppDeactivation (RhodesService.java:1496)
  at com.rhomobile.rhodes.BaseActivity.activityStopped (BaseActivity.java:186)
  at com.rhomobile.rhodes.BaseActivity.onActivityStopped (BaseActivity.java:161)
  at com.rhomobile.rhodes.BaseActivity.onStop (BaseActivity.java:263)
  at com.rhomobile.rhodes.RhodesActivity.onStop (RhodesActivity.java:510)
  at android.app.Instrumentation.callActivityOnStop (Instrumentation.java:1432)
  at android.app.Activity.performStop (Activity.java:7656)
  at android.app.ActivityThread.callActivityOnStop (ActivityThread.java:4378)
  at android.app.ActivityThread.performStopActivityInner (ActivityThread.java:4356)
  at android.app.ActivityThread.handleStopActivity (ActivityThread.java:4431)
  at android.app.servertransaction.StopActivityItem.execute (StopActivityItem.java:41)
  at android.app.servertransaction.TransactionExecutor.executeLifecycleState (TransactionExecutor.java:145)
  at android.app.servertransaction.TransactionExecutor.execute (TransactionExecutor.java:70)
  at android.app.ActivityThread$H.handleMessage (ActivityThread.java:1947)
  at android.os.Handler.dispatchMessage (Handler.java:106)
  at android.os.Looper.loop (Looper.java:214)
  at android.app.ActivityThread.main (ActivityThread.java:7037)
  at java.lang.reflect.Method.invoke (Native method)
  at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run (RuntimeInit.java:494)
  at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:965)
"main" prio=5 tid=1 Native
  | group="main" sCount=1 dsCount=0 obj=0x7489c000 self=0xefb85400
  | sysTid=4808 nice=-10 cgrp=default sched=0/0 handle=0xf281b534
  | state=S schedstat=( 610312014 16731815 1177 ) utm=34 stm=27 core=0 HZ=100
  | stack=0xff76c000-0xff76e000 stackSize=8MB
  | held mutexes=
  #00  pc 0000000000017418  /system/lib/libc.so (syscall+28)
  #01  pc 00000000000b7059  /system/lib/libart.so (_ZN3art17ConditionVariable16WaitHoldingLocksEPNS_6ThreadE+92)
  #02  pc 000000000027d38d  /system/lib/libart.so (_ZN3art3JNI9NewStringEP7_JNIEnvPKti+216)
  #03  pc 00000000018d9afd  /data/app/com.android.chrome-1/base.apk (???)
  at org.chromium.android_webview.AwContents.nativeInit (Native method)
  at org.chromium.android_webview.AwContents.<init> (PG:77)
  at vn.run (PG:15)
  at aKZ.a (PG:13)
  at aLa.run (unavailable:2)
  at org.chromium.base.task.PostTask.b (PG:13)
  at aKZ.a (PG:7)
  at com.android.webview.chromium.WebViewChromiumFactoryProvider.b (PG:6)
  at com.android.webview.chromium.WebViewChromium.init (PG:105)
  at android.webkit.WebView.<init> (WebView.java:636)
  at android.webkit.WebView.<init> (WebView.java:572)
  at android.webkit.WebView.<init> (WebView.java:555)
  at android.webkit.WebView.<init> (WebView.java:542)
  at android.webkit.WebView.<init> (WebView.java:532)
  at com.rhomobile.rhodes.webview.GoogleWebView.<init> (GoogleWebView.java:44)
  at com.rhomobile.rhodes.extmanager.RhoExtManagerImpl.createWebView (RhoExtManagerImpl.java:335)
  at com.rhomobile.rhodes.extmanager.RhoExtManagerImpl.onCreateActivity (RhoExtManagerImpl.java:912)
  at com.rhomobile.rhodes.RhodesActivity.onCreate (RhodesActivity.java:326)
  at android.app.Activity.performCreate (Activity.java:6679)
  at android.app.Instrumentation.callActivityOnCreate (Instrumentation.java:1118)
  at android.app.ActivityThread.performLaunchActivity (ActivityThread.java:2618)
  at android.app.ActivityThread.handleLaunchActivity (ActivityThread.java:2726)
  at android.app.ActivityThread.-wrap12 (ActivityThread.java)
  at android.app.ActivityThread$H.handleMessage (ActivityThread.java:1477)
  at android.os.Handler.dispatchMessage (Handler.java:102)
  at android.os.Looper.loop (Looper.java:154)
  at android.app.ActivityThread.main (ActivityThread.java:6119)
  at java.lang.reflect.Method.invoke! (Native method)
  at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run (ZygoteInit.java:969)
  at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:859)
 #00  pc 0000000000239318  /data/app/com.app.name-cZQS9eON4CdqOFC40tNGbQ==/lib/arm/librhodes.so (js_s_NewORM_useNewOrm(std::string const&, rho::json::CJSONArray&, std::string const&, std::string const&, std::string const&)+452)

This is the most reported error, it always happens, even with rhodes 6, but it doesn't crash the app, although now it happens a lot more.

java.lang.NullPointerException: 
  at com.rhomobile.rhodes.RhodesActivity.safeGetInstance (RhodesActivity.java:624)
  at com.rhomobile.rhodes.RhodesService$12.run (RhodesService.java:1619)
  at android.os.Handler.handleCallback (Handler.java:873)
  at android.os.Handler.dispatchMessage (Handler.java:99)
  at android.os.Looper.loop (Looper.java:214)
  at android.app.ActivityThread.main (ActivityThread.java:7097)
  at java.lang.reflect.Method.invoke (Native Method)
  at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run (RuntimeInit.java:494)
  at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:975)
rognar commented 5 years ago

Leonardo,

If Wonder Sistemas is willing to get the fast and reliable support please email me to krybas@tau-technologies.com

Regards, Konstantin CEO at Tau Technologies Inc.

alex-epifanoff commented 5 years ago

jfyi, SQLite is backwards compatible. The data loss may occur for encrypted database since page size was changed. To preserve old key you need to enable use_deprecated_encryption in build.yml.

alex-epifanoff commented 5 years ago

Do you have any specific steps to reproduce ANRs/crashes?

leonardofalk commented 5 years ago

jfyi, SQLite is backwards compatible. The data loss may occur for encrypted database since page size was changed. To preserve old key you need to enable use_deprecated_encryption in build.yml.

Ok, I'll try that.

Do you have any specific steps to reproduce ANRs/crashes?

No, for now all that I know about LibCurl errors is that we use the following extensions, which may be related:

I'll try to isolate and reproduce errors on a new app.

alex-epifanoff commented 5 years ago

00 pc 0000000000017418 /system/lib/libc.so (syscall+28)

01 pc 00000000000b7059 /system/lib/libart.so (_ZN3art17ConditionVariable16WaitHoldingLocksEPNS_6ThreadE+92)

02 pc 000000000027d38d /system/lib/libart.so (_ZN3art3JNI9NewStringEP7_JNIEnvPKti+216)

03 pc 00000000018d9afd /data/app/com.android.chrome-1/base.apk (???)

As you can see this happens in Chrome. Probably related to system WebView.

alex-epifanoff commented 5 years ago

java.lang.NullPointerException: at com.rhomobile.rhodes.RhodesActivity.safeGetInstance (RhodesActivity.java:624)

See https://github.com/rhomobile/rhodes/blob/master/platform/android/Rhodes/src/com/rhomobile/rhodes/RhodesService.java#L1624

As you can see it gets catched and reported to the log. It can't lead to crash or ANR.

alex-epifanoff commented 5 years ago

@leonardofalk do you have any update on this or any comments to my reply?

leonardofalk commented 5 years ago

@leonardofalk do you have any update on this or any comments to my reply?

We're currently selecting a few customers for a new test run.

00 pc 0000000000017418 /system/lib/libc.so (syscall+28)

1 pc 00000000000b7059 /system/lib/libart.so (_ZN3art17ConditionVariable16WaitHoldingLocksEPNS_6ThreadE+92)

2 pc 000000000027d38d /system/lib/libart.so (_ZN3art3JNI9NewStringEP7_JNIEnvPKti+216)

03 pc 00000000018d9afd /data/app/com.android.chrome-1/base.apk (???)

As you can see this happens in Chrome. Probably related to system WebView.

If I could tell Rhodes to ship its own WebView I'd be ok with that, but I still see this as a problem with Rhodes.

java.lang.NullPointerException: at com.rhomobile.rhodes.RhodesActivity.safeGetInstance (RhodesActivity.java:624)

See https://github.com/rhomobile/rhodes/blob/master/platform/android/Rhodes/src/com/rhomobile/rhodes/RhodesService.java#L1624

As you can see it gets catched and reported to the log. It can't lead to crash or ANR.

Well, it shouldn't be an ANR or Crash. I have three more with different backtrace, all apparently related with WebView:

java.lang.NullPointerException: 
  at com.rhomobile.rhodes.RhodesActivity.safeGetInstance (RhodesActivity.java:624)
  at com.rhomobile.rhodes.extmanager.RhoExtManagerImpl.onNavigateComplete (RhoExtManagerImpl.java:464)
  at com.rhomobile.rhodes.webview.RhoWebViewClient.onPageFinished (RhoWebViewClient.java:209)
  at uU.d (PG:307)
  at aIV.handleMessage (PG:73)
  at android.os.Handler.dispatchMessage (Handler.java:106)
  at android.os.Looper.loop (Looper.java:214)
  at android.app.ActivityThread.main (ActivityThread.java:7156)
  at java.lang.reflect.Method.invoke (Native Method)
  at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run (RuntimeInit.java:494)
  at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:975)
java.lang.NullPointerException: 
  at com.rhomobile.rhodes.RhodesActivity.safeGetInstance (RhodesActivity.java:624)
  at com.rhomobile.rhodes.extmanager.RhoExtManagerImpl.makeDefExtData (RhoExtManagerImpl.java:64)
  at com.rhomobile.rhodes.extmanager.RhoExtManagerImpl.onBeforeNavigate (RhoExtManagerImpl.java:422)
  at com.rhomobile.rhodes.webview.RhoWebViewClient.shouldOverrideUrlLoading (RhoWebViewClient.java:161)
  at android.webkit.WebViewClient.shouldOverrideUrlLoading (WebViewClient.java:73)
  at aML.a (PG:1)
  at uU.a (PG:21)
  at org.chromium.android_webview.AwContentsClientBridge.shouldOverrideUrlLoading (PG:160)
  at android.os.MessageQueue.nativePollOnce (Native Method)
  at android.os.MessageQueue.next (MessageQueue.java:328)
  at android.os.Looper.loop (Looper.java:148)
  at android.app.ActivityThread.main (ActivityThread.java:6251)
  at java.lang.reflect.Method.invoke (Native Method)
  at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run (ZygoteInit.java:1063)
  at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:924)
java.lang.NullPointerException: 
  at com.rhomobile.rhodes.RhodesActivity.safeGetInstance (RhodesActivity.java:624)
  at com.rhomobile.rhodes.extmanager.RhoExtManagerImpl.makeDefExtData (RhoExtManagerImpl.java:64)
  at com.rhomobile.rhodes.extmanager.RhoExtManagerImpl.onLoadResource (RhoExtManagerImpl.java:438)
  at com.rhomobile.rhodes.webview.RhoWebViewClient.onLoadResource (RhoWebViewClient.java:125)
  at uU.b (PG:236)
  at aIV.handleMessage (PG:15)
  at android.os.Handler.dispatchMessage (Handler.java:102)
  at android.os.Looper.loop (Looper.java:154)
  at android.app.ActivityThread.main (ActivityThread.java:6123)
  at java.lang.reflect.Method.invoke (Native Method)
  at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run (ZygoteInit.java:867)
  at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:757)

Also, when I last checked there was this segmentation fault error, that corresponds to roughly 90~% of the crash reports (and not the NPE I've mentioned earlier)

pc 000000000056b81c  /data/app/com.wonder_sistemas.probusm2-2/lib/arm/librhodes.so
alex-epifanoff commented 5 years ago

We're going to make some changes and publish a new build this week. Issues can probably be related to new CLANG toolchain introduced in newer NDKs.

alex-epifanoff commented 5 years ago

Can you check if this resolves your crashes https://github.com/rhomobile/rhodes/commit/e6c5225636d0087503ee06a5cd8daa3971b810e5

leonardofalk commented 5 years ago

Can you check if this resolves your crashes e6c5225

Ok, testing...

jfyi, SQLite is backwards compatible. The data loss may occur for encrypted database since page size was changed. To preserve old key you need to enable use_deprecated_encryption in build.yml.

It didn't work, I just went from rhodes 7 to rhodes 6 build and it deletes everything from database.

Here's my build.yml

---
## Note: sdk is full path to Rhodes root. If not defined - automatically find latest installed Rhodes gem
#sdk: "C:/rhodes"
sdkversion: 7.1.0
name: ProbusM2
version: "16.538"
vendor: Wonder Sistemas
## Note: rho_app_id send to RhoConnect server for identify application (used in multi-push configuration only)
## we recommend make it same with BundleIdentifier
rho_app_id: com.wondersistemas.probusm2
build: release
applog: rholog.txt

# enable encrypt of database
# encrypt_database: 1
use_deprecated_encryption: 1

# enable encryption of bundle by AES (only for iOS and Adnroid)
# key should be generated by this command:
# $ rake build:bundle:generate_AES_key
#encrypt_files_key: "9kefijsPZjfOHDWNhpKiiq3bbewg5rgv83Hb/qMAQSw="
#encrypt_file_extensions:
#  - html
#  - js
#  - css
#  - iseq

## Note: enbale it for use new style of iOS top status bar(battery info, etc.) - status bar is transparency and over application window
#iphone_use_new_ios7_status_bar_style: 1

## Note: enable this for force iOS full screen (top status bar with battery info etc. is hidden)
#iphone_full_screen: 1

## Note: enable specific device capabilities for aplication
capabilities:
  ## allows use of audio hardware (record audio, modify audio settings)
  #- audio
  ## allows use of hardware camera
  - camera
  ## allows use of geo location service
  - gps
  ## allows read device network state (connected/disconnected)
  - network_state
  ## allow make calls/read phone state
  #- phone
  ## allow read/modify personal information and contacts
  #- pim
  ## allows use of PUSH on device
  #- push
  ## allows use of hardware vibration mechanism
  #- vibrate
  ## allows use of bluetooth hardware
  - bluetooth
  ## allows access to default device calendar
  #- calendar
  ## allows writing to the SD card on Android devices
  - sdcard
  ## esse nao ta documentado mas eh autoexplicativo
  - hardware_acceleration

application-icon-dir: c:/mobile/client/icon

uwp:
  productid: '9caeef00-91f1-0137-b036-080027574d48'
  target: 'ARM'

iphone:
  configuration: Release
  ## emulatortarget can be "iphone", "ipad" or UUID or Name of already exist simulator
  emulatortarget: iPhone X
  ## Note: sdk can be 3 predefined value : latest, latest_simulator, latest_device
  ## simulator or device detect by rake command for "rake run:iphone" is simulator, for "rake device:iphone:production" is device
  ## sdk also can be defined exactly like:  iphonesimulator10.2 or iphoneos10.2
  sdk: latest
  ## Note: Provisioning
  ## There are two provisioning type for modern XCode: Automatic and Manual
  ## development_team must be configured for any type provisioning
  ## you see team code by run this command:
  ## $ security find-identity -p codesigning -v
  ## you will see list of certificates and you should copy code from "iPhone Distribution" line like this (code is PNQRMU3LPM):
  ## 5) D6D5EE35B3F8D9C04E85C7015539F04B38403DA3 "iPhone Distribution: Vasya Pupkin (PNQRMU3LPM)"
  ##
  ## Note: This is Automatic provisioning configuration:
  #provisioning_style: Automatic
  #development_team: "PNQRMU3LPM"
  ##
  ## Note: This is Manual provisioning configuration:
  ## provisionprofile can be defined as name or UUID - if you defined provisionprofile then codesignidentity defined automatically if you not defined it in this build.yml
  #provisionprofile: "Current WildCart 05"
  #provisioning_style: Manual
  #development_team: "PNQRMU3LPM"
  #codesignidentity: "iPhone Developer"
  ##
  #entitlements:
  BundleIdentifier: com.wondersistemas.probusm2
  BundleURLScheme: probusm2
  ## Note: Specifies the URL schemes the app is able to call using Rho.System.runApplication
  #ApplicationQueriesSchemes: ['scheme1', 'scheme2']
  ## Note: Specifies list of HTTP domains for connect from application
  #http_connection_domains: ['domain1.com', 'domain2.org']
  ## Note: Specify remove of Info.plist keys - use this if you need to remove some keys from Info.plist
  ## This example remove two keys with unsupported screen orientation
  #info_plist_data_remove:
  #- UISupportedInterfaceOrientations: ['UIInterfaceOrientationLandscapeLeft', 'UIInterfaceOrientationLandscapeRight']
  ## Note: Specifies set/modify of Info.plist keys with content(hash) - all of this items will be apply to final Info.plist of application. You can specify any Ino.plist data in this parameter.
  ## This example allow all HTTP connections in application:
  #info_plist_data:
  #  NSAppTransportSecurity:
  #    NSAllowsArbitraryLoads: true
  ## if you use "barcode" extension on iOS, you can switch barcode recognize engine: APPLE_BARCODE_ENGINE(default), ZXING, ZBAR
  #barcode_engine: APPLE_BARCODE_ENGINE
  extensions:
    - mediacapture
    - zlib
    - devicesecurity

android:
  manifest_template: AndroidManifest.erb

  abis:
    - arm
    # - aarch64
    - x86

  # specifies compile SDK version ( 9 corresponds to API 28). If not specified then latest available API level will be used.
  version: 5
  minSDK: 21
  maxSDK: 27
  targetSDK: 26

  ## Note: in order to simplify debugging only app messages and system channels with priority informative and higher, and any errors are enabled by default
  logcatFilter: APP:I StrictMode:I DEBUG:I *:E
  extensions:
    ## extensão para manter a tela ligada na sincronizacao
    # apenas android pois nao compila no VisualStudio
    - wutil
    - mediacapture
    - zlib
    - devicesecurity

## Note: setup android_title to 0 for remove default window titlebar in Android application
## Note: set this to 0 for remove title bar of Android application
## also setup "full_screen" in rhoconfig.txt
android_title: 0

wm:
  #sdk: "MC3000c50b (ARMV4I)"
  #sdk: "Windows Mobile 6.5.3 Professional DTK (ARMV4I)"
  extensions:
    - webkit

win32:
  deployqt: 1
  deploymsvc: 1
  msvc: 2015

sailfish:
  configuration: Release
  target: 'arm'
  target_sdk: 'SailfishOS-2.1.4.13-armv7hl'
  build_threads: 1
  enable_web_engine: false
  device:
    type: real
    device_name: INOI R7 (ARM)
    user: nemo
    key: C:/SailfishOS/vmshare/ssh/private_keys/INOI_R7_(ARM)/nemo
    password: 123
    host: '192.168.2.15'
  extensions:
    - mediacapture
    - barcode
    - signature
    - rhoconnect-client
    - serialport

extensions:
  - coreapi
  - set
  - net-http
  - json
  - fileutils
  - mspec
  - uri
  - timeout
  - digest
  - digest-md5
  - thread

rhobundle:
  exclude_items:
  - thumb.db

# Rhosimulator no macOS
env:
  paths:
    rhosimulator: /Users/wonder/projects/rhodes/platform/osx/bin/RhoSimulator
alex-epifanoff commented 5 years ago

You don't have an encrypted DB judging from the build.yml so no sense to use encryprion-specific settings. If you just migrated 5.x->6.x->7.x with unencrypted DB it should not drop data. You may want to check the logfiles. We can check it you have specific steps and sample code to reproduce.

leonardofalk commented 5 years ago

You don't have an encrypted DB judging from the build.yml so no sense to use encryprion-specific settings. If you just migrated 5.x->6.x->7.x with unencrypted DB it should not drop data. You may want to check the logfiles. We can check it you have specific steps and sample code to reproduce.

The problem is downgrading, we went from 6 -> 7, then downgraded from 7 -> 6 because of the other issues I've mentioned, then we lost data.

alex-epifanoff commented 5 years ago

https://www.sqlite.org/formatchng.html

In other words, since 2004 all SQLite releases have been backwards compatible, though not necessarily forwards compatible.

leonardofalk commented 4 years ago

Hello, sorry for the long reply, we've been releasing incremental builds and it took a while to reach a good number of clients, I just wanted to let you know that is fine now and thank you for your help and support!