Deletes Highlights

[Bergene]

Hi.

I have experienced this issue multiple times today. When adding the latest highlights from my clippings file it somehow deleted existing clipping from "Warbreaker" Page and inserted "Oathbringer" highlights into the "Warbreaker" page.

It seems to have been solved by deleting all pages in "My Clipping" database and running the script on My Clippings file. Do the script require "My Clippings" to not contain any existing Highlights?

[paperboi]

Um sorry, but I need more context here. Do you have two clipping files on your device? Can you explain your device model and a general clipping of the books as such?

[theneo13]

This is something that I have noticed too. What happened is, I have added new notes to the book 'Deep Work'. I ran the script, to update the same. But what happened is, my notes from the book 'Complete Work of' ( actual name is Complete works of Swami Vivekananda, not sure why it is reading just that part ) got deleted and this updated highlights of Deep Work got added to 'Complete Works of'

Edit: What I have observed is right before the first highlight of the 'Deep Work', there is 'Complete Works of ' . So all the notes of 'Complete Works' got deleted and updated notes of 'Deep Work' got added there, leaving 'Deep Work' with old highlights.

[Bergene]

That is the same issue as I am experiencing. I actually have two clippings files on my device. One is called "My Clippings.txt" and the other is named "My Clippings_archived.txt". Can the archived clippings be causing the issues even though they don't have the same name? I can try to remove it and see if it works better after that. I have attached a sample Clippings file with the Warbreaker hightlights that are replaced by the Oathbringer hightlights. My Kindle device is Kindle Paperwhite WiFi 2018

Clippings Sample.txt

[theneo13]

Today I have added new highlights to one more book( The psychology of money ). I ran the scrip again. This time 'Complete Works of' got added again....now I have 2 of those and the new highlights of the book ' The psychology of money ' got added to 2nd Complete Works of', but they did not get updated in ' The psychology of money '

[Bergene]

I have now tried to remove the additional Clippings file, no change.

The deletion and replacement of highlights have not happened since yesterday but I still have an issue with the same books every time I try to add new highlights to the Notion Database.

This is the output after I have deleted the pages from the Notion Database and successfully transferring my Notes and Highlights to Notion (whole output included in kindle2notion_output - empty database.txt):

Notion page is found. Analyzing clippings file...
Found 163 notes and highlights.

× Passed 2 bookmarks or unsupported clippings.

Initiating transfer...

.........

Oathbringer (The Stormlight Archive) (Brandon Sanderson)
--------------------------------------------------------
✓ Added book cover
✓ 42 notes / highlights added successfully

Warbreaker (Brandon Sanderson)
------------------------------
✓ Added book cover
✓ 2 notes / highlights added successfully

Transfer complete... Exiting script...

Then I added a new note from Oathbringer to "My Clippings" file and ran your script (kindle2notion) again. No I get this output:

Notion page is found. Analyzing clippings file...
Found 164 notes and highlights.

× Passed 2 bookmarks or unsupported clippings.

Initiating transfer...

Oathbringer (The Stormlight Archive) (Brandon Sanderson)
--------------------------------------------------------
✓ 41 notes / highlights added successfully

Warbreaker (Brandon Sanderson)
------------------------------
✓ -41 notes / highlights added successfully

Transfer complete... Exiting script...

---

I have now deleted the Oathbringer page from Notion. It transfers all hightlights (including the new one) to Notion successfully now that there are no records for that book.

I have tested with some other books also. When updating an already existing page for a specific book I get output like this:

Notion page is found. Analyzing clippings file...
Found 169 notes and highlights.

× Passed 2 bookmarks or unsupported clippings.

Initiating transfer...

Halo: Silentium (Greg Bear)
---------------------------
✓ -37 notes / highlights added successfully

Mortal Dictata (Karen Traviss)
------------------------------
✓ -1 notes / highlights added successfully

Oathbringer (The Stormlight Archive) (Brandon Sanderson)
--------------------------------------------------------
✓ 38 notes / highlights added successfully

Transfer complete... Exiting script...

For Halo Silentium I have added two highlights. For Halo Silentium I have added two highlights. For Oathbringer I have added no highlights nor notes. And it should be 43 highlights/notes, not 38.

---

I ran it again for laughs and I got this output:

Notion page is found. Analyzing clippings file...
Found 169 notes and highlights.

× Passed 2 bookmarks or unsupported clippings.

Initiating transfer...

Halo: Silentium (Greg Bear)
---------------------------
✓ -3 notes / highlights added successfully

Mortal Dictata (Karen Traviss)
------------------------------
✓ -1 notes / highlights added successfully

Words of Radiance: 3 (The Stormlight Archive) (Brandon Sanderson)
-----------------------------------------------------------------
✓ 4 notes / highlights added successfully

Transfer complete... Exiting script...

It is the same Clippings file, no changes made to it since last time I ran the script. As last time; For Halo Silentium I have added two highlights. As last time; For Halo Silentium I have added two highlights. For Words of Radiance I have added no highlights nor notes. And it should be 9 highlights/notes, not 4.

After deleting the page for the specific books that have new Highlights the issues are gone.

---

Workaround: For now I will be deleting the Pages in Notion for the book I am currently reading everytime I want to update the Highlights/Notes on Notion. The issue I can see with this workaround is if I do not have all the clippings data available anymore and started with a fresh blank Clippings file I will have to delete the page from Notion and then update it with my new Hightlight.. but I will not have my old highlights unless I manually back these up in Notion.

I hope this isn't to much or too little data :P

[paperboi]

I guess having a JSON structure to keep track of each book and the clippings under them would help. I'll need more time to think of other possible solutions and implement it. Will update this thread in the coming days.

[Bergene]

Thank you. :) I love the idea. Keep it up.

[theneo13]

Well, I have deleted the book( which I am currently reading ) from Notion. Then ran the script. The book got added with updated highlights without any hiccups. But 'The Complete Works of ' got added again. Every single time I run the script, the same book in my case 'The Complete Works of' is getting added.

[Bergene]

For me it helped to delete the troublesome Notion Pages. Have you tried to delete the Notion page for "The Complete Works of" and adding it to a fresh page?

[theneo13]

Yes as of now, that's the workaround.

[paperboi]

Thank you. :)

I love the idea. Keep it up.

Thank you for raising the issue and providing so much information regarding it.

@theneo13 Ditto to you too :)

[Bergene]

Hi.

I have not experienced the issue where the script replaces Highlights from one book with highlights from another since I opened this issue. The issue where the script does not update the notion database still happens though. I have not experienced this issue the last few days but today it just popped up again. It seems to be the same two books all the time. I had to delete the Notion pages for Oathbringer and Warbreaker, then it updated all my notes and highlights to Notion successfully.