Error uploading highlights

[manikoutai]

After installing version 1.0.2, forllowing the resolution of the issue #40 , I was able to fully access my Kobo and configure october with the readwise token.

When I tried to upload the highlights I received the following error:

There was a problem sending your highlights: Received a non-200 status code from Readwise: code 400

Although I tried it, I was scared to import so many hightlights without having the control about which books to select from. That would be a great feature.

Windows 10.

[marcus-crane]

Hi there,

400 definitely indicates that something has gone wrong on the client side (ie; not with Readwise's servers) but we'll need to pinpoint what exactly.

I just tested with my own highlights and they synced fine on Windows so it may be the case that a highlight isn't being handled properly.

October generates some logs that should hopefully pinpoint the issue so if you could provide those that'd be great. You're welcome to send them to october@utf9k.net and I can have a look to see where the upload failed and see if I can replicate it.

The quickest way to find your logs is to open the Run dialog (by searching it in the start menu) and entering%LOCALAPPDATA%\october\logs

It should open an explorer window with logs labelled with today's date. If you just send through the latest one, that should have the information we're after.

As far as the ability to control what books to select, that's definitely what I'd like to work towards. I had that in mind at first but it increases the scope of the application quite a bit so I started out with something simple first and from here, we can iterate towards a more fleshed out UI.

That said, I'll make a Github Issue to track it going forward (edit: #45)

Thanks!

[marcus-crane]

Ah, I should also mention that it's Easter Weekend coming up tomorrow and I don't plan to take a laptop with me so I'll have a look at anything you're able to send through before the weekend starts but otherwise I might not be able to look at this issue for a few days. In the meantime, you can always build more highlights 😉

[Kerfufflez]

Hey Marcus, I also just got this error 400 trying to sync my notes from my side-loaded kobo. I'm running an m1 mac. Can you point me to where the log files are? If you're cool with it, I'll send the log files over for you to look at.

[marcus-crane]

@Kerfufflez Hey there, you can find the macOS logs at /Users/<you>/Library/Application Support/october/logs described here: https://github.com/marcus-crane/october#issues

You can jump there by opening Finder and selecting Go -> Go to Folder from the top menu and entering in the path above

I'd like to make it easier to export these logs directly through October in future

[Kerfufflez]

@marcus-crane Got it, just emailed! Thank you sir

[marcus-crane]

Based on the logs provided from @Kerfufflez, bookmarks are detected by October but it fails to correlate those bookmarks with the books they originated from.

As a result, it tries to submit an empty list to Readwise which is an invalid action.

It may be due to formatting or other epub quirks, I can't really say just yet but it's a start anyway.

[marcus-crane]

As a bit of background, there is a Bookmark table on the Kobo database and each row contains a link to a book, a highlight and/or an annotation.

There is another table called Content which contains a whole bunch of stuff. It isn't just say; a list of books but it may contain say; an entry for every chapter. Only one of those duplicates has the information we need like the author, the title and so on.

October has to basically do some rummaging around to piece together all of the information we need to submit a highlight along with the book it came from.

It's essentially failing to find all the information it needs and (incorrectly) throwing up its hands and ending up with something not quite correct.

It's not surprising, there are many book formats and some may or may not have all the data needed so while there are a couple of checks in place, it'll probably take some time to figure out all the edge cases

[Kerfufflez]

This is great, thank you for looking into it so quick. I will also try to look into those tables and see if I can find anything.

[Varming73]

I have the same problem, any progress on this?

[marcus-crane]

I have the same problem, any progress on this?

Hi there,

This issue does pop up from time to time for some users and it's something I generally have to review on a case by case basis.

A handful of times users ran into this, it was due to having one highlight that was longer than Readwise's upload limit. The latest release adds a mechanism for splitting up excessively long (ie; more than 8000 character) highlights.

Anyway, I'd need some more information to try and debug that cause of the issue you're facing.

As a start, if you're on macOS, there are some log files generated that will be able to help pinpoint the issue.

There are instructions earlier in this thread about how to find them and I can review them for you if you send them through to october@utf9k.net

Unfortunately there's a bug currently with the Windows build where logs are not properly generated and I haven't gotten around to fixing it as I don't use Windows too often myself

If you are on Windows (or if the logs don't show much for macOS), I can also run your Kobo database through a debug build, see at which point it crashes and then see about issuing a fix.

Anyway, if you are on macOS, the logs are a good start if you're able to send those to october@utf9k.net so I can start investigating