[Bug] Suwayomi Database storing old metadata (bato source)

Kickunio commented 1 month ago

PLEASE READ THIS

I acknowledge that:

I have updated to the latest version of the app.
I have tried the troubleshooting guide described in README.md
If this is a request for adding/changing an extension it should be brought up to your extension repo.
If this is an issue with some extension not working properly, It does work inside Tachiyomi as intended.
I have searched the existing issues and this is a new ticket NOT a duplicate or related to another open issue
I will fill out the title and the information in this template

Note that the issue will be automatically closed if you do not fill out the title or requested information.

Device information

Suwayomi-Server version: v1.0.0-r1527-docker)
Server Operating System: docker
Server Desktop Environment: docker
Server JVM version: docker
Client Operating System: docker
Client Web Browser: Chrome (newest version)

Steps to reproduce

Open bato library
Some thumbnails don't show up when browsing the source

Expected behavior

All thumbnails should be loaded while browsing library

Actual behavior

Some thumbnails don't load - they error out and even reply doesn't work. To reload a picture you have to open the manga/manhwa with the thumbnail/cover problem and refresh it from inside - then it works correctly It seems the server tries to connect to a CDN server that doesn't exist anymore:

log:

18:08:39.439 [DefaultDispatcher-worker-9] ERROR suwayomi.tachidesk.server.JavalinSetup -- IOException while handling the request

java.io.IOException: xfs-s117.batcg.org: Name or service not known at suwayomi.tachidesk.manga.impl.Manga.fetchHttpSourceMangaThumbnail(Manga.kt:291) at suwayomi.tachidesk.manga.impl.Manga.fetchHttpSourceMangaThumbnail$default(Manga.kt:271) at suwayomi.tachidesk.manga.impl.Manga$fetchMangaThumbnail$2.invokeSuspend(Manga.kt:320) at suwayomi.tachidesk.manga.impl.Manga$fetchMangaThumbnail$2.invoke(Manga.kt) at suwayomi.tachidesk.manga.impl.Manga$fetchMangaThumbnail$2.invoke(Manga.kt) at suwayomi.tachidesk.manga.impl.util.storage.ImageResponse.getImageResponse(ImageResponse.kt:70) at suwayomi.tachidesk.manga.impl.Manga.fetchMangaThumbnail(Manga.kt:319) at suwayomi.tachidesk.manga.impl.Manga.getMangaThumbnail(Manga.kt:365) at suwayomi.tachidesk.manga.controller.MangaController$thumbnail$2$1.invokeSuspend(MangaController.kt:94) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108) at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115) at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684) Caused by: java.net.UnknownHostException: xfs-s117.batcg.org: Name or service not known at java.base/java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method) at java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(Unknown Source) at java.base/java.net.InetAddress.getAddressesFromNameService(Unknown Source) at java.base/java.net.InetAddress$NameServiceAddresses.get(Unknown Source) at java.base/java.net.InetAddress.getAllByName0(Unknown Source) at java.base/java.net.InetAddress.getAllByName(Unknown Source) at java.base/java.net.InetAddress.getAllByName(Unknown Source) at okhttp3.Dns$Companion$DnsSystem.lookup(Dns.kt:49) at okhttp3.internal.connection.RouteSelector.resetNextInetSocketAddress(RouteSelector.kt:169) at okhttp3.internal.connection.RouteSelector.nextProxy(RouteSelector.kt:132) at okhttp3.internal.connection.RouteSelector.next(RouteSelector.kt:74) at okhttp3.internal.connection.RealRoutePlanner.planConnect(RealRoutePlanner.kt:147) at okhttp3.internal.connection.RealRoutePlanner.plan(RealRoutePlanner.kt:67) at okhttp3.internal.connection.FastFallbackExchangeFinder.launchTcpConnect(FastFallbackExchangeFinder.kt:118) at okhttp3.internal.connection.FastFallbackExchangeFinder.find(FastFallbackExchangeFinder.kt:62) at okhttp3.internal.connection.RealCall.initExchange$okhttp(RealCall.kt:267) at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:32) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:84) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:65) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at eu.kanade.tachiyomi.network.interceptor.CloudflareInterceptor.intercept(CloudflareInterceptor.kt:42) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at eu.kanade.tachiyomi.network.interceptor.UserAgentInterceptor.intercept(UserAgentInterceptor.kt:19) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at eu.kanade.tachiyomi.network.interceptor.UncaughtExceptionInterceptor.intercept(UncaughtExceptionInterceptor.kt:18) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:205) at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:537) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source)

EDIT: The problem is connected with Suwayomi storing old metadata in the database file. For now the only fix is to backup your manga database and restore it on a fresh database

Robonau commented 1 month ago

duplicate of #714

Kickunio commented 1 month ago

duplicate of #714

Yes and No at the same time. In #714 the problem was that it shown error 410 and here the domain doesn't exist (DNS_PROBE_FINISHED_NXDOMAIN in chrome when browsing the CDN address)

Robonau commented 1 month ago

then its not an issue we introduce its an extension issue

Kickunio commented 1 month ago

This is not extension based issue - Even using Tachimanga on iOS (suwayomi based) all thumbnails load without problem using the same extension. And the problem fixes when I refresh the manga using refresh button in the manga details. After that the thumbnail and cover load up correctly - sometimes it even loads more chapters than there were before clicking refresh. It looks like it looks up some old data and doesn't reload the manga list.

Robonau commented 1 month ago

oh, you have a previously cached thumbnail that is borked, clear the server cache (and client cache)

Kickunio commented 1 month ago

If you mean the clear server cache button – then I did that (also cleared website cache in chrome) but it didn’t change a thing. The only change is that I had to refresh thumbnails and covers in the library (not the source) (as it also didn’t load correctly without refreshing the manga manually)

Robonau commented 1 month ago

are you not running category/library updates or do they not update bato manga?

Kickunio commented 1 month ago

The library is updating normally for bato (for the library entries) but when I browse the source it doesn't load the thumbnails/covers until I refresh the manga manually. And if I clear the cache most of bato-based library entries have a missing thumbnail/cover until I refresh each manga manually.

Robonau commented 1 month ago

set Automatically refresh metadata to true (at /settings/librarySettings on webui) then run a mass update

Kickunio commented 1 month ago

Ok, I tested the problem on the fresh instance and everything is OK. Maybe the clear cache button doesn't clear the cache all the way? Is there any other way to clear the cache?

Robonau commented 1 month ago

its not the cache its that the thumbnail remote url is dead and mass updates by default don't update metadata (think title, description, thumbnail, etc) just doing what i said above, setting Automatically refresh metadata to true and mass updating should work fine

Kickunio commented 1 month ago

Ok, that fixed the library problem. But the thumbnail problem while browsing the source still persists. On the other hand, on the new test instance there is no problem.

Robonau commented 1 month ago

yea, all the data you loaded previously for all the manga on that source still persists so all the broken urls are still there there is currently no fix for this

Kickunio commented 1 month ago

What I can confirm after another experiment: This is not container based-problem because I changed the mapped folder for the original container with the new test container folder and the problem with broken urls doesn't persist. Something breaks the the old mapped folder - not permissions (they are the same between the folders), so maybe some file is locked out ?

Robonau commented 1 month ago

yea, its a database storing old metadata thing, the only ways to fix it (right now) are manually editing the Database, or making a backup, deleting the DB, and restoring on a fresh remake, else wait for it to be fixed

Kickunio commented 1 month ago

Good to know - just done that without knowing that :). I'll change the name of the bug to show the correct problem.

Suwayomi / Suwayomi-Server