search

woocommerce / woocommerce-android

WooCommerce Android app
https://www.woocommerce.com/mobile
GNU General Public License v2.0
277 stars 135 forks source link

(FluxC) Crash: Row too big to fit into CursorWindow #572

Open jkmassel opened 5 years ago

jkmassel commented 5 years ago

This one comes from http://crashes.to/s/01951aa4300

Relevant bits:

Fatal Exception: org.greenrobot.eventbus.EventBusException: Invoking subscriber failed
Caused by android.database.sqlite.SQLiteBlobTooBigException: Row too big to fit into CursorWindow requiredPos=0, totalRows=13
AmandaRiu commented 5 years ago

Additional Info:

Fatal Exception: org.greenrobot.eventbus.EventBusException: Invoking subscriber failed
       at org.greenrobot.eventbus.EventBus.handleSubscriberException(EventBus.java:527)
       at org.greenrobot.eventbus.EventBus.invokeSubscriber(EventBus.java:509)
       at org.greenrobot.eventbus.EventBus.invokeSubscriber(EventBus.java:501)
       at org.greenrobot.eventbus.HandlerPoster.handleMessage(HandlerPoster.java:67)
       at android.os.Handler.dispatchMessage(Handler.java:106)
       at android.os.Looper.loop(Looper.java:193)
       at android.app.ActivityThread.main(ActivityThread.java:6863)
       at java.lang.reflect.Method.invoke(Method.java)
       at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:537)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)

Caused by android.database.sqlite.SQLiteBlobTooBigException: Row too big to fit into CursorWindow requiredPos=0, totalRows=13
       at android.database.sqlite.SQLiteConnection.nativeExecuteForCursorWindow(SQLiteConnection.java)
       at android.database.sqlite.SQLiteConnection.executeForCursorWindow(SQLiteConnection.java:859)
       at android.database.sqlite.SQLiteSession.executeForCursorWindow(SQLiteSession.java:836)
       at android.database.sqlite.SQLiteQuery.fillWindow(SQLiteQuery.java:62)
       at android.database.sqlite.SQLiteCursor.fillWindow(SQLiteCursor.java:149)
       at android.database.sqlite.SQLiteCursor.getCount(SQLiteCursor.java:137)
       at android.database.AbstractCursor.moveToPosition(AbstractCursor.java:220)
       at android.database.AbstractCursor.moveToNext(AbstractCursor.java:269)
       at com.yarolegovich.wellsql.SelectQuery.getAsModel(SelectQuery.java:154)
       at com.yarolegovich.wellsql.SelectQuery.getAsModel(SelectQuery.java:147)
       at org.wordpress.android.fluxc.persistence.OrderSqlUtils.getOrdersForSite(OrderSqlUtils.kt:62)
       at org.wordpress.android.fluxc.store.WCOrderStore.getOrdersForSite(WCOrderStore.kt:158)
       at com.woocommerce.android.ui.orders.OrderListPresenter.fetchAndLoadOrdersFromDb(OrderListPresenter.kt:127)
       at com.woocommerce.android.ui.orders.OrderListPresenter.onOrderChanged(OrderListPresenter.kt:96)
       at java.lang.reflect.Method.invoke(Method.java)
       at org.greenrobot.eventbus.EventBus.invokeSubscriber(EventBus.java:507)
       at org.greenrobot.eventbus.EventBus.invokeSubscriber(EventBus.java:501)
       at org.greenrobot.eventbus.HandlerPoster.handleMessage(HandlerPoster.java:67)
       at android.os.Handler.dispatchMessage(Handler.java:106)
       at android.os.Looper.loop(Looper.java:193)
       at android.app.ActivityThread.main(ActivityThread.java:6863)
       at java.lang.reflect.Method.invoke(Method.java)
       at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:537)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)
nbradbury commented 5 years ago

Note: this is likely due to the 2MB limit on cursor rows.

nbradbury commented 5 years ago

Fixing this would be a pretty big task and we have no reports of this crash this month, so I'm going to mark this as "wontfix" but not close it.

jkmassel commented 5 years ago

We had another instance of this!

AmandaRiu commented 5 years ago

Removing from main project, but keeping in the groundskeeping project so it can be fixed in the next round.

sentry-io[bot] commented 5 years ago

Sentry issue: WOOCOMMERCE-ANDROID-2Z

nbradbury commented 5 years ago

Related WPAndroid issue: https://github.com/wordpress-mobile/WordPress-Android/issues/10057

nbradbury commented 5 years ago

We've had two more users experience this crash this month.

nbradbury commented 5 years ago

Interesting - it turns out since Android P we can request a larger cursor size:

if (cursor is SQLiteCursor && Build.VERSION.SDK_INT >= Build.VERSION_CODES.P) {
   (cursor as SQLiteCursor).window = CursorWindow(null, (1024 * 1024 * 10).toLong())
}

Not sure how we'd do this with WellSql, though.

nbradbury commented 4 years ago

Possible fix for this over in WellSql repo.

AmandaRiu commented 4 years ago

This FluxC PR updates to the latest version of WellSql where we're testing out enlarging the SQLBlob size in DEBUG builds before we push it to production builds.

sentry-io[bot] commented 4 years ago

Sentry issue: WOOCOMMERCE-ANDROID-C4

nbradbury commented 4 years ago

Closing this as #2497 hopefully addresses the issue for those running Android P and above.

astralbodies commented 4 years ago

@AmandaRiu Can you take a look and see if the crashes that are still coming in for this exception? If I'm understanding the PR it should have been introduced in WCAndroid 4.4 and is still happening.

AmandaRiu commented 4 years ago

@astralbodies Actually, the fix is only for debug builds so it definitely wouldn't have fixed anything for production. We were going to run it in debug builds for a couple releases to make sure it doesn't cause any issues before promoting it to production. The fix will only work for Android P BTW, but it's definitely an improvement. I'll open a PR to remove the debug flag.

astralbodies commented 4 years ago

@AmandaRiu That makes sense! Can we keep this issue open then until that fix is build for production?

AmandaRiu commented 4 years ago

@astralbodies absolutely!

AmandaRiu commented 4 years ago

Reopening this ticket as the crash is still happening on devices running 28+. The most recent crash happened on Android 11:

android.database.sqlite.SQLiteBlobTooBigException: Row too big to fit into CursorWindow requiredPos=16, totalRows=17
    at android.database.sqlite.SQLiteConnection.nativeExecuteForCursorWindow(SQLiteConnection.java)
    at android.database.sqlite.SQLiteConnection.executeForCursorWindow(SQLiteConnection.java:1001)
    at android.database.sqlite.SQLiteSession.executeForCursorWindow(SQLiteSession.java:838)
    at android.database.sqlite.SQLiteQuery.fillWindow(SQLiteQuery.java:62)
    at android.database.sqlite.SQLiteCursor.fillWindow(SQLiteCursor.java:161)
    at android.database.sqlite.SQLiteCursor.onMove(SQLiteCursor.java:131)
    at android.database.AbstractCursor.moveToPosition(AbstractCursor.java:248)
    at android.database.AbstractCursor.moveToNext(AbstractCursor.java:280)
    at com.yarolegovich.wellsql.SelectQuery.getAsModel(SelectQuery.java:158)
    at com.yarolegovich.wellsql.SelectQuery.getAsModel(SelectQuery.java:151)
    at org.wordpress.android.fluxc.persistence.OrderSqlUtils.getOrdersForSiteByRemoteIds(OrderSqlUtils.kt:122)
    at org.wordpress.android.fluxc.store.WCOrderStore.getOrdersForDescriptor(WCOrderStore.kt:337)
    at com.woocommerce.android.ui.orders.list.OrderListItemDataSource.getItemsAndFetchIfNecessary(OrderListItemDataSource.kt:50)
    at com.woocommerce.android.ui.orders.list.OrderListItemDataSource.getItemsAndFetchIfNecessary(OrderListItemDataSource.kt:38)
    at org.wordpress.android.fluxc.model.list.datasource.InternalPagedListDataSource.getItemsInRange(InternalPagedListDataSource.kt:51)
    at org.wordpress.android.fluxc.model.list.PagedListPositionalDataSource.loadRangeInternal(PagedListFactory.kt:58)
    at org.wordpress.android.fluxc.model.list.PagedListPositionalDataSource.loadInitial(PagedListFactory.kt:40)
    at androidx.paging.PositionalDataSource.dispatchLoadInitial(PositionalDataSource.java:286)
    at androidx.paging.TiledPagedList.<init>(TiledPagedList.java:107)
    at androidx.paging.PagedList.create(PagedList.java:229)
    at androidx.paging.PagedList$Builder.build(PagedList.java:388)
    at androidx.paging.LivePagedListBuilder$1.compute(LivePagedListBuilder.java:206)
    at androidx.paging.LivePagedListBuilder$1.compute(LivePagedListBuilder.java:171)
    at androidx.lifecycle.ComputableLiveData$2.run(ComputableLiveData.java:101)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
    at java.lang.Thread.run(Thread.java:923)
sentry-io[bot] commented 4 years ago

Sentry issue: WOOCOMMERCE-ANDROID-XV

nbradbury commented 3 years ago

We're still seeing these crashes (505 events, 10 users). In this FluxC PR, a similar problem in WPAndroid was addressed by not storing raw JSON in the database. It's possible that's our problem, too, since we store JSON in the order table, but note that WPAndroid continues to have these crashes despite that fix.

nbradbury commented 3 years ago

Not sure why we didn't notice this before, but it appears that the crash happens most often in WCOrderStore.hasCachedOrdersForSite(). We load the entire list of a site's orders in that method, but we really only need to do a count of the orders. I'll submit a FluxC issue and PR to address this.

nbradbury commented 3 years ago

Looking at Sentry, these exceptions are happening most frequently in hasCachedOrdersForSite, and also quite often in getItemsAndFetchIfNecessary. They also occurred a few times in getProductList and once in fetchOutdatedOrders.

Perhaps we could fix the issue with getItemsAndFetchIfNecessary by using a more basic order model for the order list, since it displays a small subset of order properties?

nbradbury commented 3 years ago

@anitaa1990 I'm going to re-open this since #3189 only fixes the crash in one area. We still see crashes when loading orders, which wasn't addressed by that PR.