search

wallabag / android-app

Android application to read your articles saved in your wallabag. You can also easily add new articles.
https://www.wallabag.org
GNU General Public License v3.0
483 stars 259 forks source link

Recently released update breaks all sync #1049

Closed shikasta-net closed 4 years ago

shikasta-net commented 4 years ago

Issue details

After updating to 2.4.0 of the app, my sync fails part way through.

Duplicate?

I don't see a report of this since the update was released a few days ago.

Actual behaviour

A full update, or even wiping the database, results in a crash part way through import with the error Unknown error com.squareup.moshi.JsonDataException: Expected a string but was BEGIN_ARRAY at path $._embedded.items[11].published_by[0] Stacktrace: com.squareup.moshi.JsonDataException: Expected a string but was BEGIN_ARRAY at path $._embedded.items[11].published_by[0] at com.squareup.moshi.JsonUtf8Reader.nextString(JsonUtf8Reader.java:644) at com.squareup.moshi.StandardJsonAdapters$10.fromJson(StandardJsonAdapters.java:215) at com.squareup.moshi.StandardJsonAdapters$10.fromJson(StandardJsonAdapters.java:213) at com.squareup.moshi.JsonAdapter$2.fromJson(JsonAdapter.java:137) at com.squareup.moshi.CollectionJsonAdapter.fromJson(CollectionJsonAdapter.java:76) at com.squareup.moshi.CollectionJsonAdapter$2.fromJson(CollectionJsonAdapter.java:53) at com.squareup.moshi.JsonAdapter$2.fromJson(JsonAdapter.java:137) at com.squareup.moshi.ClassJsonAdapter$FieldBinding.read(ClassJsonAdapter.java:194) at com.squareup.moshi.ClassJsonAdapter.fromJson(ClassJsonAdapter.java:156) at com.squareup.moshi.JsonAdapter$2.fromJson(JsonAdapter.java:137) at com.squareup.moshi.CollectionJsonAdapter.fromJson(CollectionJsonAdapter.java:76) at com.squareup.moshi.CollectionJsonAdapter$2.fromJson(CollectionJsonAdapter.java:53) at com.squareup.moshi.JsonAdapter$2.fromJson(JsonAdapter.java:137) at com.squareup.moshi.ClassJsonAdapter$FieldBinding.read(ClassJsonAdapter.java:194) at com.squareup.moshi.ClassJsonAdapter.fromJson(ClassJsonAdapter.java:156) at com.squareup.moshi.JsonAdapter$2.fromJson(JsonAdapter.java:137) at com.squareup.moshi.ClassJsonAdapter$FieldBinding.read(ClassJsonAdapter.java:194) at com.squareup.moshi.ClassJsonAdapter.fromJson(ClassJsonAdapter.java:156) at com.squareup.moshi.JsonAdapter$2.fromJson(JsonAdapter.java:137) at retrofit2.converter.moshi.MoshiResponseBodyConverter.convert(MoshiResponseBodyConverter.java:45) at retrofit2.converter.moshi.MoshiResponseBodyConverter.convert(MoshiResponseBodyConverter.java:27) at retrofit2.OkHttpCall.parseResponse(OkHttpCall.java:225) at retrofit2.OkHttpCall.execute(OkHttpCall.java:188) at retrofit2.DefaultCallAdapterFactory$ExecutorCallbackCall.execute(DefaultCallAdapterFactory.java:104) at wallabag.apiwrapper.WallabagService.execAndCheckBody(WallabagService.java:1499) at wallabag.apiwrapper.WallabagService.lambda$getArticles$0$WallabagService(WallabagService.java:218) at wallabag.apiwrapper.-$$Lambda$WallabagService$nylw3DAgOM4zumOAh8m95sasuhg.run(Unknown Source:4) at wallabag.apiwrapper.NotFoundPolicy.call(NotFoundPolicy.java:140) at wallabag.apiwrapper.NotFoundPolicy.call(NotFoundPolicy.java:133) at wallabag.apiwrapper.NotFoundPolicy.call(NotFoundPolicy.java:128) at wallabag.apiwrapper.WallabagService.getArticles(WallabagService.java:218) at wallabag.apiwrapper.ArticlesQueryBuilder.execute(ArticlesQueryBuilder.java:282) at wallabag.apiwrapper.ArticlesPageIterator.hasNext(ArticlesPageIterator.java:80) at fr.gaulupeau.apps.Poche.service.workers.ArticleUpdater.performUpdate(ArticleUpdater.java:183) at fr.gaulupeau.apps.Poche.service.workers.ArticleUpdater.update(ArticleUpdater.java:127) at fr.gaulupeau.apps.Poche.service.workers.ArticleUpdateWorker.updateArticles(ArticleUpdateWorker.java:77) at fr.gaulupeau.apps.Poche.service.workers.ArticleUpdateWorker.update(ArticleUpdateWorker.java:37) at fr.gaulupeau.apps.Poche.service.tasks.UpdateArticlesTask.run(UpdateArticlesTask.java:17) at fr.gaulupeau.apps.Poche.service.tasks.ActionRequestTask.run(ActionRequestTask.java:25) at fr.gaulupeau.apps.Poche.service.-$$Lambda$Nf0ej7UkvM-IGhcyH2GG-UaljDo.run(Unknown Source:2) at fr.gaulupeau.apps.Poche.service.TaskService.run(TaskService.java:154) at fr.gaulupeau.apps.Poche.service.TaskService.lambda$gjfracnqY8x0_-mrxsx0oqjQmgk(Unknown Source:0) at fr.gaulupeau.apps.Poche.service.-$$Lambda$TaskService$gjfracnqY8x0_-mrxsx0oqjQmgk.run(Unknown Source:2) at java.lang.Thread.run(Thread.java:764)

Expected behaviour

Syncing completes even if there were errors?

Environment details

Logs

wallabag server

Nothing visible in the server log but it's inside docker so I may not have found it.

Web server

Just a sequence of GET /api/entries.json up to page 39 and then stops.

Your experience with wallabag Android app

All worked fine before the update.

di72nn commented 4 years ago

https://github.com/wallabag/android-app/issues/887#issuecomment-570032818.

shikasta-net commented 4 years ago

Running select id, url, published_by from wallabag_entry where published_by like '%;a:%'; finds one entry, 1308 | https://www.elitedaily.com/women/why-its-hard-to-talk-feelings/1023633 | a:1:{i:0;a:1:{i:0;s:9:"Eve Stern";}} but re-fetching content doesn't repair it.

shikasta-net commented 4 years ago

Manually replacing a:1:{i:0;a:1:{i:0;s:9:"Eve Stern";}} with a:1:{i:0;s:9:"Eve Stern";} in the database fixes the issue. Is there a way to auto-repair these (or handle them in the app)?

di72nn commented 4 years ago

Is there a way to auto-repair these

No, it's not a very common problem.

or handle them in the app

Maybe in the future.

Server 2.3.8 fetched this article without authors, so I guess the problem is a remnant of old parsing.

re-fetching content doesn't repair it

Maybe re-fetching doesn't update authors if none found. I didn't know that.

anarcat commented 4 years ago

wait is this fixed? because it's breaking updates for me, which is especially problematic when setting up a new device. in that case, the full sync totally fails and the device doesn't get any articles at all... shouldn't the app at least handle the exception?

equivalent issue on the server side https://github.com/wallabag/wallabag/issues/4411

di72nn commented 4 years ago

wait is this fixed?

@anarcat it's not really a client issue - the server contains incorrect data.

shouldn't the app at least handle the exception?

It doesn't crash, does it? The app presents you with a searchable error message. It's not perfect, but it is a bit problematic to fix in the app. This problem should die down eventually when everyone fixes their old problematic articles.

This comment explains how to fix it.

anarcat commented 4 years ago

It doesn't crash, does it? The app presents you with a searchable error message.

The sync crashes: it completely fails to download any content, and makes the entire app unusable.

It's not perfect, but it is a bit problematic to fix in the app. This problem should die down eventually when everyone fixes their old problematic articles.

Maybe a server-side upgrade should fix those kind of brokenness?

di72nn commented 4 years ago

Maybe a server-side upgrade should fix those kind of brokenness?

Sure, but it's not up to me.

anarcat commented 4 years ago

Sure, but it's not up to me.

Fair enough. At least we have https://github.com/wallabag/wallabag/issues/4411 that we can hope get fixed on the server side...