Excessive looper threads with RealmExtensionFlowable

[bricestacey]

I am building a fairly large application using this library. We've noticed on older devices that performance is poor. We identified the problem to be that the RealmExtensionsFlowable file creates a new thread and looper for every subscription if it's not already on the main thread. This causes a large number of threads with loopers, all contending for limited resources. To better understand scope -- we're using Nexus 5 and Nexus 6P as poor performing devices with 10s-20s lockups. Pixel 2 you don't really notice anything.

As a short-term solution, I forked the repo and replaced the thread creation with a singleton thread with a looper. All the performance issues go away. You can see the commit here https://github.com/bricestacey/Kotlin-Realm-Extensions/commit/e07ebab2e2507ccc85046e1eeb6d8b5de6d13af9

I'm curious what we might be able to do to allow this behavior to be configurable. Off the top of my head

1) Add a parameter to the flowable extensions to specify a scheduler 2) Refactored to use some default LooperFactory, then expose a means of configuring it

I kind of think both are desirable. 1 is sufficient, but 2 would allow setting a default case or more complex strategies. I have only been using Kotlin a few months and this is the only extension library I'm using outside the core android one. I haven't seen anything yet for how to configure them so don't have much insight.

Happy to take a stab at this, but wanted to bring it up before I started down some haphazard path.

[vicpinm]

Hi @bricestacey , thanks so much for your report. It's true that a new looper thread is created when you perform a flowable operation, but that thread should be finished once you have finished your subscription. Check this code from prepareObservableQuery() method in RealmExtensionsFlowable class:

   .doOnCancel {
                realm?.close()
                mySubscription?.dispose()
                if (isRealmThread()) {
                    looper?.thread?.interrupt()
                }
            }

Could you check if this code is executed in your case when you dispose your subscription? Maybe your problem is related with disposing your subscriptions. I haven't noticed any performance issues on devices older than nexus 5, like moto g 2014.

Also, you can avoid thread creation if you execute your flowable operations on a looper thread you own, like main thread or any other thread with a looper attached to it.

[bricestacey]

Hey, thanks for the follow up. The threads do get cleaned up properly. The library is behaving correct as far as I know and our code is correctly disposing itself.

We actually identified the problem further to be slow queries on the main thread. We've since removed the check that the Flowable is already on a thread with a Looper. Instead, we always execute the queries on a single background thread with a looper.

I can investigate further whether a single thread is necessary or if it is okay to have many.

Also, you can avoid thread creation if you execute your flowable operations on a looper thread you own, like main thread or any other thread with a looper attached to it.

Absolutely, but since downstream Flowables can override our subscribeOn calls, we'd need to wrap our Flowables with a Flowable.defer just as this library has done. It adds some overhead both to execute the code and also boilerplate.

I'd prefer if there were some way to configure a SchedulerFactory for the entire extension or add an optional scheduler parameter to the extension. I wanted to propose the idea before opening a PR or whatnot.

[Zhuinden]

Yeah, using copyFromRealm() on UI thread has no guarantees of being fast, because it eager-loads entire colleciton.