janeczku / calibre-web

:books: Web app for browsing, reading and downloading eBooks stored in a Calibre database
GNU General Public License v3.0
13.22k stars 1.41k forks source link

Kobo Sync not working #2339

Closed br4mmy closed 10 months ago

br4mmy commented 2 years ago

I got a remote server and Calibre-web is reachable on http://mydomainname.com/calibreweb. Calibreweb runs on port 8083 and sits behind nginx:

location /calibreweb {
        proxy_bind              $server_addr;
        proxy_pass              http://127.0.0.1:8083;
        proxy_set_header        Host            $http_host;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header        X-Scheme        $scheme;
        proxy_set_header        X-Script-Name   /calibreweb;  # IMPORTANT: path has NO trailing slash
}

I am not using Docker.

I configured Calibre-web but one thing is not working: Kobo Sync Generated a Kobo Auth Url: api_endpoint=https://mydomainname.com/calibreweb/kobo/api-key Enabled remote log in ("Magic link") Enabled Kobo sync Enabled Proxy unknown requests to Kobo store Server external port (API calls): 80 (as default)

When I use the browser on pc to access this link, than I get a page with only {}

I have adjusted the kobo/Kobo eReader.conf with api_endpoint=https://mydomainname.com/calibreweb/kobo/api-key but after ejecting the Kobo and restarting it, I get "synchronisation" failed.

Anything that I should have done to get this working? Sorry for this possibly noobish question...

renobles commented 2 years ago

I have a very similar issue. Just slight difference in config. My calibreweb instance is at a subdomain, not a subfolder. Also, my server external port is 443. When I try to sync, it just gives the synchronization failed error. Here's what I see in the calibre-web logs:

[2022-03-18 09:16:45,064] INFO {cps.kobo:140} Kobo library sync request received. [2022-03-18 09:16:45,064] DEBUG {cps.kobo:141} SyncToken: [very long token here] 2022-03-18 09:16:45,093] DEBUG {cps.kobo:215} Books to Sync: 0 [2022-03-18 09:16:45,114] DEBUG {cps.kobo:280} Remaining books to Sync: 0 [2022-03-18 09:16:55,335] ERROR {cps.kobo:340} Failed to receive or parse response from Kobo's sync endpoint: HTTPSConnectionPool(host='storeapi.kobo.com', port=443): Read timed out. (read timeout=10)

daugvinasr commented 2 years ago

Try http://mydomainname.com/calibreweb/kobo/{token}/v1/library/sync in the pc browser. Do you see the books then? Remember to force full kobo sync, because if the token was used before it will not output full information again unless there is an update to the library.

renobles commented 2 years ago

So I've made a bit of progress, but still not a solid sync. I tested the URL above (modified to my setup) and did get book data. Then forced a full kobo sync, and turned on debug logging on the Kobo. This led me to see it was getting stalled out in the sync process before actually trying to sync books. I then adjusted my proxy settings to not run calibre-web through my auth frontend, and instead use the built in authentication mechanism, as I suspected the sync process was trying to pull from something in the calibre-web path that I hadn't bypassed already. All I had in my auth bypass was /kobo. Exempting the whole thing got the Kobo much farther in the sync process.

Now what I see is that when it gets to the "SyncLibraryCommand" step in the sync, it fails with a 502 error - bad gateway. This is the url that gives the error: https://calibre-web.{mydomain}.com/kobo/{token}/v1/library/sync?Filter=ALL&DownloadUrlFilter=Generic,Android&PrioritizeRecentReads=true

Going to the same URL in my PC's browser does not return an error. Interestingly, Calibre Web seems to think the sync was successful though, as the return data is what it looks like after a sync has taken place. I have to reset the Kobo sync data and refresh to see the list of books again.

OzzieIsaacs commented 2 years ago

This response is the largest one. I think the payload is to big. Some users had similiar issues, please try to increase the ‚max body size‘ parameter in your proxy config. this would also fit to: calibre-web thinks the sync was successful, as data got lost later on

renobles commented 2 years ago

Success! It wasn't the max body size parameter, as that was already unlimited for my config. That did lead me to look at the nginx logs though and I saw this each time I tried to sync:

upstream sent too big header while reading response header from upstream

So after some research I increased the size of the proxy headers in my proxy.conf:

proxy_busy_buffers_size   1024k;
proxy_buffers   4 512k;
proxy_buffer_size   1024k;

Now I may have gone overkill on what they need to be, but this worked and now I have a complete sync. Thank you!

kchopein commented 1 year ago

This is also happeing to me. I see there's a way to work around the issue... but is there a way to fix it, so others like me don't face this issue in the future? What do you think, should this addressed in this repo or in the Docker image repo? How can I help, not being a Python developer?

mendhak commented 1 year ago

I've added @renobles example to additional hints in the Wiki, because it worked for me too.