Cached data is not shown if no internet connection

[arthabus]

Hi, thanks for the great library.

I've integrated it recently in my project according to the instructions and it seemed to work fine. But I've noticed that if no internet connections, the cached data often doesn't show up - the StoreService.getAll returns an empty list.

Though in rear cases it shows the data again (after several attempts of restarting the app) but then it disappears again. So the data is cached for sure.

Any advice on what might be wrong here?

Here is the key parts of the code if it helps:

gradle:

    implementation 'com.github.playmoweb:store2realm:3.0.2'

    implementation "com.squareup.retrofit2:retrofit:2.3.0"
    implementation "com.squareup.retrofit2:adapter-rxjava2:2.3.0"
    implementation "com.squareup.retrofit2:converter-gson:2.3.0"
    implementation "com.squareup.okhttp3:okhttp:3.4.1"
    implementation 'com.squareup.okhttp3:logging-interceptor:3.4.1'
    implementation "org.parceler:parceler-api:1.1.5"
    annotationProcessor "org.parceler:parceler:1.1.5"
    implementation 'io.reactivex.rxjava2:rxandroid:2.0.1'

Model:

@Parcel(implementations = arrayOf(SchoolRealmProxy::class),
        value = Parcel.Serialization.BEAN,
        analyze = arrayOf(School::class))

open class School() : RealmObject(), HasId, Parcelable {

    @PrimaryKey
    var id: String? = ""; //4
    var name: String? = null; //1
....
}

ApiService:

interface ApiService {

    @get:GET("/api/schools")
    val schoolList: Flowable<List<School>>

    object Creator {

        fun buildApiService(): ApiService {
            val interceptor = HttpLoggingInterceptor()
            interceptor.level = HttpLoggingInterceptor.Level.BODY

            val clientOkHttp = OkHttpClient.Builder()
                    .addInterceptor(interceptor)
                    .build()

            val gson = GsonBuilder()
                    .registerTypeAdapter(object : TypeToken<RealmList<PageItem>>() {}.type,
                            PageItemRealmListConverter())
                    .create()

            val retrofit = Retrofit.Builder()
                    .baseUrl(BuildConfig.API_BASE_URL)
                    .client(clientOkHttp)
                    .addConverterFactory(GsonConverterFactory.create(gson))
                    .addCallAdapterFactory(RxJava2CallAdapterFactory.create())
                    .build()

            return retrofit.create(ApiService::class.java)
        }
    }
}

SchoolService:

public class SchoolService extends StoreService<School> {

    public SchoolService(ApiService apiService) {
        super(School.class, new SchoolDao(apiService));
        this.syncWith(new BaseRealmService<>(School.class));
    }

    public static class SchoolDao extends StoreDao<School> {
        private final ApiService apiService;

        public SchoolDao(ApiService apiService) {
            this.apiService = apiService;
        }

        @Override
        public Flowable<Optional<List<School>>> getAll(Filter filter, SortingMode sortingMode) {
            return wrapOptional(apiService.getSchoolList());
        }
    }
}

Usage:

public class DataProviderJava {
    private static DataProviderJava sInstance = new DataProviderJava();

    private CompositeDisposable disposable = new CompositeDisposable();

    public static DataProviderJava single(){
        return sInstance;
    }

    private DataProviderJava(){
        mApiService = ApiService.Creator.INSTANCE.buildApiService();
        mSchoolService = new SchoolService(mApiService);
   }
    public void getSchoolList(DataListener<List<School>> listener) {
        getData(mSchoolService.getAll(), listener);
    }
    private <T> void getData(Flowable<Optional<T>> flowable, DataListener<T> listener){

        Disposable subscriber = flowable
                .subscribeOn(Schedulers.io())
                .observeOn(AndroidSchedulers.mainThread())
                .subscribeWith(new DisposableSubscriber<Optional<T>>() {
                    @Override
                    public void onNext(Optional<T> data) {
                        listener.onSuccess(data.get());
                    }

                    @Override
                    public void onError(Throwable t) {
                        listener.onError(t);
                    }

                    @Override
                    public void onComplete() {
                    }
                });
        disposable.add(subscriber);
    }

    private void loadData() {
        showProgress();

        DataProviderJava.single().getSchoolList(new DataProviderJava.DataListener<List<School>>() {
            @Override
            public void onSuccess(List<School> data) { 
                hideProgress();
                setupDropDown(data); <- this 'data' is often an empty list if no internet connection
            }

            @Override
            public void onError(Throwable t) {
                hideProgress();
            }
        });
    }

Thanks

[tspoke]

Hi @ArtemBogush , thank you for using store2realm.

I will have a look as soon as possible into this issue, thank you for your code ! Do you have any deadlines ? Is it really blocking for you for now ? I'll try to look into this issue tonight or tomorrow.

[arthabus]

Hi @tspoke, thanks, appreciate that very much! We are still in development phase but we have to decide on how to proceed with the caching functionality and in case we have to implement our system instead, we might be quite tight on schedule as this library really helped us a lot.

So basically it depends and if the issue could be fixed during the next week or so we are still good. If not, then probably we have to dedicate resources to implement internal caching to make sure we still can meet the deadline.

Feel free to ping me here or drop me an email at artem.bogush.work@gmail.com if I can provide any additional details - I'll be glad to cooperate on this.

And thanks again for the great library and for sharing it with the community!

[tspoke]

I found the reason of this.

The reason : the Http request fail faster/during the realm computation, even if the conception of Store2Store (behind Store2Realm lib) dispatch data in the good order, the two 'calls' to the two services (realm & your api) are done in parallel and if one failed it breaks the full chain of syncing.

Your case show a situation where one node of the graph of Stores is unreachable (no network); an error is throw and is catch into the onError() callback, stopping all the Rx chain. It's not really a bug, because this library has not been made to work as a offline cache, but more as a library to sync different sources (named Stores). What I want to say is that all the Stores should be available to allow syncing, and the cache is only used to show data before update by another Store (network latency for instance). Empty Store != unavailable store.

The previous, and old, version of Store2Store handled this situation because it was built as a cache not as a syncing system.

I understand the issue behind this concept, I'm gonna improve that to handle differently this case, using better Rx objects. But the problem is that today's version of Store2Store allow you to sync different Store like one API to another API (both with latency) and adding the concept of cache (fallback & fast response) is something slightly different.

For now

That said, for your project, and for now, you can handle network issues as follow :

postService.getAll()
       .onErrorResumeNext(new Function<Throwable, Flowable<Optional<List<Post>>>>() {
               @Override
               public Flowable<Optional<List<Post>>> apply(Throwable throwable){
                    // you can do what you want here, like update UI depending on the typeof throwable
                    return Flowable.just(new Optional<List<Post>>(null)); // return an empty Optional
               }
        })
        .subscribeOn(Schedulers.io())
        .observeOn(AndroidSchedulers.mainThread())
        .subscribeWith(new DisposableSubscriber<Optional<List<Post>>>(){
                @Override
                public void onNext(Optional<List<Post>> items) {
                    if(!items.isNull()){ // if not empty, show them
                        mainView.updatePosts(items.get());
                    }
                }
                ...
         });

There is a case where you will probably need to know if the UI has been refreshed at least one time (by the realm cache or by the API); you can do something like this :

            new DisposableSubscriber<Optional<List<Post>>>(){
                    private boolean hasBeenUpdated = false; // local boolean

                    @Override
                    public void onNext(Optional<List<Post>> items) {
                        if(!items.isNull()){
                            mainView.updatePosts(items.get());
                            hasBeenUpdated = true;
                        }
                    }

                    @Override
                    public void onComplete() {
                        if(!hasBeenUpdated){
                            // yourUI.showEmptyView();
                        }
                    }
             }

Be careful, do not try to move the onErrorResumeNext() method in the DAO class or service class. If you return an empty list for instance, Store2Store will think it's a legal object to be synced and will erase the local copy (realm) with your empty List. The network error should be catch in the Presenter to allow you to display information in the user without interfering with the inner syncing process.

Upcoming update

I will ship a new version of Store2Store & Store2Realm to improve the library and handle this case. It will take some days, if not weeks (I hope not, depending on my personal time). For now, I think you should be good to go with the code above. Do not hesitate to give me feedback or your opinion on this :)

[arthabus]

@tspoke, thanks, awesome! That works.

Just added onErrorResumeNext that returns null optional as per your response and now the local data is available even if no internet connection.

Thanks again for your help and for the great library!

[tspoke]

No problems @ArtemBogush, do not hesitate to open an other issue if you need help again :)

[tspoke]

@ArtemBogush

I just merged some code. After digging a little bit to allow cache with this library I found a 'special and unknown' argument in rxjava.

You can remove the onErrorResumeNext and do something like this instead :

postService.getAll(filter, sortingMode)
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedulers.mainThread(), true) // add true here if you want to delay the error
    .subscribeWith(new DisposableSubscriber<Optional<List<Post>>>(){
        @Override
        public void onNext(Optional<List<Post>> items) {
            if(!items.isNull()){
                mainView.updatePosts(items.get());
            }
        }

        @Override
        public void onError(Throwable e) {
            mainView.showError("Network error");
        }

        @Override
        public void onComplete() {
        }
    });

This boolean will force the subscriber to receive event in order of subscribing (and the realm store will emits items before the no-network-error). Then, if there is no network, the error will be received in the onError. If no errors have been emitted, the onComplete will be called.

You can test the example module in the project ;) !

Have fun !

[arthabus]

@tspoke wow, thanks a lot for the update! That definitely will work.