Kobo Sync: Unable to download books on kobo reader

[MikeOlsen]

After updating the Kobo eReader.conf, I was able to sync. The metadata was visible, but not the cover images and I was unable to download the books.

I noticed after opening the Kobo eReader.conf that the following url:s had been updated to my servers ip address image_host, image_url_quality_template and image_url_template. But the port was missing, so after adding the port I managed to get the cover images working.

[OzzieIsaacs]

I can confirm that the urls are changed in the file (in my reader they are changed to ::::[port]/[...]. I didn't realize this before. Could you please tell us which Kobo device you tested. Did you activate the "proxy unknown requests to kobo"? What does the calibre-web log say. Would it be possible for you to change the log level to debug and also activate the access log. As the sync mechanism is only based on knowing the string in the api-endpoint could you please replace this string with something different and post the content. Last question what is the exact behaviour for not being able to download the book?

Side remarks: It should only show epub and kepub files and only download this two types of books.

[MikeOlsen]

I'm using a Clara HD running version 4.19.14123 (12/10/19).

I have tried both with and without the "proxy unknown requests to kobo", I also tried changing the port to 80 on the server. With no luck.

Do you have any other attributes in your config that the kobo has changed?

Here is part of my config:

[OneStoreServices]
account_page=https://secure.kobobooks.com/profile
account_page_rakuten=https://my.rakuten.co.jp/
annotations_host=https://annotations.kobo.com
api_endpoint=http://192.168.0.5/kobo/[token]
autocomplete=https://storeapi.kobo.com/v1/products/autocomplete
book=https://storeapi.kobo.com/v1/products/books/{ProductId}
book_detail_page=https://store.kobobooks.com/{culture}/ebook/{slug}
book_detail_page_rakuten=http://books.rakuten.co.jp/rk/{crossrevisionid}
book_landing_page=https://store.kobobooks.com/ebooks
categories=https://storeapi.kobo.com/v1/categories
category=https://storeapi.kobo.com/v1/categories/{CategoryId}
category_products=https://storeapi.kobo.com/v1/categories/{CategoryId}/products
dictionary_host=https://kbdownload1-a.akamaihd.net
discovery_host=https://discovery.kobobooks.com
dropbox_link_account_poll=
eula_page="https://www.kobo.com/termsofuse?style=onestore"
featured_list=https://storeapi.kobo.com/v1/products/featured/{FeaturedListId}
featured_lists=https://storeapi.kobo.com/v1/products/featured
feedback=
fte_feedback=https://storeapi.kobo.com/v1/products/ftefeedback
image_host=http://192.168.0.5
image_url_quality_template=http://192.168.0.5/kobo/[token]/{ImageId}/image.jpg
image_url_template=http://192.168.0.5/kobo/[token]/{ImageId}/image.jpg
kobo_dropbox_link_account_enabled=
kobo_nativeborrow_enabled=True
kobo_redeem_enabled=True
kobo_subscriptions_enabled=False
kobo_superpoints_enabled=False
kobo_wishlist_enabled=True
love_data=@ByteArray(\0\0\0\0)
love_points_redemption_page="https://store.kobobooks.com/{culture}/KoboSuperPointsRedemption?productId={ProductId}"
magazine_detail_page=
magazine_landing_page=https://store.kobobooks.com/emagazines
oauth_host=https://oauth.kobo.com
password_retrieval_page=https://www.kobobooks.com/passwordretrieval.html
privacy_page="https://www.kobo.com/privacypolicy?style=onestore"
product_recommendations=https://storeapi.kobo.com/v1/products/{ProductId}/recommendations
product_reviews=https://storeapi.kobo.com/v1/products/{ProductIds}/reviews
purchase_buy_templated=https://www.kobo.com/{culture}/checkout/createpurchase/{ProductId}
registration_page="https://authorize.kobo.com/signup?returnUrl=http://store.kobobooks.com/"
review=https://storeapi.kobo.com/v1/products/reviews/{ReviewId}
review_sentiment=https://storeapi.kobo.com/v1/products/reviews/{ReviewId}/sentiment/{Sentiment}
sign_in_page="https://authorize.kobo.com/signin?returnUrl=http://store.kobobooks.com/"
social_authorization_host=https://social.kobobooks.com:8443
social_host=https://social.kobobooks.com
store_host=store.kobobooks.com
store_search="https://store.kobobooks.com/{culture}/Search?Query={query}"
subs_landing_page=
subs_management_page=
subs_plans_page=
subs_purchase_buy_templated=
taste_profile=https://storeapi.kobo.com/v1/products/tasteprofile
user_ratings=https://storeapi.kobo.com/v1/user/ratings
user_recommendations=https://storeapi.kobo.com/v1/user/recommendations
user_reviews=https://storeapi.kobo.com/v1/user/reviews
userguide_host=https://kbdownload1-a.akamaihd.net

[MikeOlsen]

Access log:

[2020-02-26 17:43:08,968]  INFO {cps.kobo:123} Kobo library sync request received.
[2020-02-26 17:43:08,968] DEBUG {cps.kobo:125} Kobo: Received unproxied request, changed request port to server port
[2020-02-26 17:43:09,312] DEBUG {cps.kobo:203} Kobo: Received unproxied request, changed request port to server port
[2020-02-26 17:43:09,312]  INFO {cps.kobo:204} Kobo library metadata request received for book e686a2a0-e4d8-4b16-a565-dc4670f0424c
[2020-02-26 17:43:09,630] DEBUG {cps.kobo:390} Unimplemented User Request received: http://192.168.0.5/kobo/[token]/v1/user/wishlist

calibre-web log:

::ffff:192.168.0.134 - - [2020-02-26 17:43:09] "GET /kobo/[token]/v1/library/sync?Filter=ALL&DownloadUrlFilter=Generic,Android&PrioritizeRecentReads=true HTTP/1.1" 200 3766 0.222471
::ffff:192.168.0.134 - - [2020-02-26 17:43:09] "GET /kobo/[token]/v1/library/e686a2a0-e4d8-4b16-a565-dc4670f0424c/metadata HTTP/1.1" 200 2999 0.010531
::ffff:192.168.0.134 - - [2020-02-26 17:43:09] "GET /kobo/[token]/v1/user/wishlist?PageSize=30&PageIndex=0 HTTP/1.1" 307 558 0.003058

As the sync mechanism is only based on knowing the string in the api-endpoint could you please replace this string with something different and post the content.

Not sure I follow, do you mean regenerating the sync token?

Last question what is the exact behaviour for not being able to download the book?

It stays at pending for a second then goes back to showing download, if I instead press read now it gives the following error message:

Download failed please try again.

Side remarks: It should only show epub and kepub files and only download this two types of books. They are of the format ePub uploaded through the web ui.

[OzzieIsaacs]

You posted the token, this was what I tried to avoid. You should delete the current one and generate a new one. If your calibre-web instance is accessible via internet and somebody knows the address he/she now can download all books. (If it would work)

[OzzieIsaacs]

From the logs it looks like your started a download: It starts the sync, -> successful It requests the book metadata -> gets answer, but kobo doesn't like the content, so download isn't started. Not a bad result, it's almost working Further investigation is depending on your computer and programming skills. Would it be possible for you to install/start Wireshark and capture the traffic during the sync? Would it be possible for you to debug the python code (I tell you where to look)? Or is it better I create a new branch with additional outputs and you change your installation do an new download and post the content?

[OzzieIsaacs]

Depending on the way you choose, it would be good to know your operating system

[OzzieIsaacs]

Ah, and I hope you are not using docker, this would complicate everything

[OzzieIsaacs]

Okay I also tried it, it's also not working for me, as I think it was last weekend. No need for you to do anything else.

[OzzieIsaacs]

You are accessing your kobo device via IPV6, the download URL generated is invalid, I'm fixing it

[MikeOlsen]

You posted the token, this was what I tried to avoid.

Don't worry its a test setup on a private network, and i also changed most of the letters :)

Ah, and I hope you are not using docker, this would complicate everything

I'm actually using the linuxserver/calibre-web docker one, which seems to be based on the master branch and not the realeas version. I noticed this when I pressed update in the web ui and it actually downgraded to the release version and the kobo integration was no longer there.

Anyhow, I can set it up on a virtual machine if you need me to help out with testing.

[OzzieIsaacs]

Fixed it. Now it's up to you to get the docker container updated. In the past linuxserver offered a latest container, which is updated on weekly base on sundays. If the update works inside the container you should be able to update to the newest commit, by switching to nightly updates (admin-> basic settings -> servers settings) and then update. Switching back to stable offers you the downgrade to the latest stable version (your experience sounds like this option)

[MikeOlsen]

Thanks so much, I really appreciate the work you're doing!

[itszaen]

Hi, is this problem resolved? I followed #1273, switched to nightly, sneaked into the container (hehe) and installed python3-jsonschema, and was able to enable the Kobo Sync feature. I am having the same problem that MikeOlson was (is?) having, the debug log says "Unimplemented User Request received". I have tried enabling the "proxy unknown requests to kobo store" for the sake of trying, no difference. Am I not updating correctly, or it's a LinuxServer issue and I'll have to wait? The LS Docker image is up-to-date by the way.

[OzzieIsaacs]

The message is not a real problem, the api is not fully implemented, so there are some unknown requests

[itszaen]

Oh, should have been clearer. I cannot download the book, cannot "read now" either. FYI, I'm using Kobo Forma with firmware version 4.20.14622

[OzzieIsaacs]

I‘m not sure if read now should work, but download should, so you can sync, see all books, but missing covers? There are some issues pending regarding cover download ( #1277). I‘m currently not sure if it should work, need some time to check

[itszaen]

The problems are still exactly what MikeOlsen reported above, to borrow his words:

It stays at pending for a second then goes back to showing download, if I instead press read now it gives the following error message: Download failed please try again.

I would post my logs, but they are exactly the same as Mike's.

Forgot to mention, I'm using the '4/18/20, 1:17 AM' version, so very bleeding edge. (Time zone is JST) Could that be the problem?

Edit: weird writing

[itszaen]

I tried it again after migrating all my books to Calibre Web, still no good :( Will just use opds for now.

[OzzieIsaacs]

My automatic test currently fails before I get to the point where it downloads the book. I remember that generating the download URLs was problematic, as KOBO creates invalid(?) http requests, the port number is missing in the request. Please be patient, I'm starting to work on the last PR regarding kobo, will create some tests about it and merge it to the master branch

[itszaen]

No worries! Thank you for all your hard works.

[uschi1]

Hallo ich habe das gleiche Problem, auch auf Docker calibre ist 0.6.20 und kann auch keine Bücher herunterladen, dass Image ist aktuell linuxserver/calibre-web

[uschi1]

Ich habe einen Kobo Aura h2o