Syncing notes and extracted annotations to Notion database

[shantanusingh16]

Hi, I'd really like to thank you for providing such an amazing plugin. I am really grateful since this simplifies my workflow by a significant extent and saves me a huge effort.

However, as part of my workflow, I do annotate and comment in the pdf files attached to every item in my zotero collection. I use the zotfile extension to extract those annotations out as a separate item. Is it possible to sync these extracted annotations to notion as well?

[dvanoni]

Hi @shantanusingh16, thanks for the kind feedback!

I think it should be possible to sync those annotations to Notion. Do you have an idea of what would be the best place to save them in Notion? Do you think they should be added directly to the body of the Notion page, or would it make sense to have each Zotero note be its own sub-page on the Notion page?

As body text	As sub-page

Unfortunately, I won't be able to work on this right away as my time is limited, but I like this idea!

[shantanusingh16]

Hi @dvanoni

I think having it in the body of the same page under the heading "Extracted Annotations" should be fine. I personally try to keep the nesting minimum to ensure we get to the required content sooner. I can look at your code and see if I can implement this. We can discuss the implementation details, on google meet or something, if you want over a call sometime.

[panyuz]

I think notes generated by user should be sync to notion database too.

[jigarp12892]

Hello! @dvanoni , I just set up this plugin and I am so happy with it!

I am here to offer my +1 to exporting both annotations and notes generated. My 2 cents toward making this scalable are : each annotation goes under the heading "annotation" and each note's content goes under the heading {note name} in the body of the page.

Thank you for making this for us. You deserve an honorary PhD for helping out us academics.

[dvanoni]

@jigarp12892, thank you for the kind feedback! 😊

I think your suggestion sounds like a great approach!

[jigarp12892]

@jigarp12892, thank you for the kind feedback! 😊

I think your suggestion sounds like a great approach!

I understand you have other commitments would like to help with anything that can make it easier for you.

[22-modifier]

This would be the exact functionality I've been looking for! Less for writing papers, but would allow my co-surgical residents and I to generate a more organized, communal annotated bibliography of key papers that guide clinical practice. Currently nothing is out there makes it easy to navigate notes linked to a bibliography and the paper itself. Thank you for the already awesome tool you've created!

[realCrush]

This would be a super nice feature, and can let people get main idea or valueble part of articles with a glance, as a PhD or researcher, this save lives when facing tens of hundreds PDF files!🤗

[theotheo]

I'm totally agree it's very useful features. I want to try to implement it. @dvanoni What do you think about it?

[dvanoni]

@theotheo, I'd be happy if you want to try implementing this!

One of the challenges I foresee with implementing this is finding a way to sync the annotations content into the Notion page without overwriting anything else that may already be there. I imagine that some folks would want to both sync their annotations from Zotero but also take additional notes (or add other content) directly in Notion.

One way I was thinking of approaching this was using a technique similar to how Notero currently correlates Zotero items with Notion pages—i.e. saving a link in Zotero with the unique URL of the Notion page. I think we might be able to use a similar approach, but instead of the page URL we'd use the ID of the block on the page containing the annotations synced from Zotero.

Then we could have logic along these lines:

• If a block ID is not saved in Zotero, assume we haven't synced annotations into Notion yet
  • Create a new block on the page and save its ID into Zotero
  • Add all the annotations content as children of the new block
• If a block ID is saved in Zotero, assume we've previously synced annotations
  • Delete all the children of the block (because I imagine it would be too challenging to update annotations in place)
  • Add all the annotations content as children of the block

I'm not sure which block type would be best for the top-level block here—perhaps a paragraph? I was thinking the new Toggle Headings they recently announced would be a nice fit for this, but unfortunately it doesn't look like they've added them to the API yet.

To be honest, I haven't worked with the blocks part of the API yet, so I really have no idea if this is the best approach. I'm curious to hear what you're thinking!

[mai-m-ai]

hi @dvanoni，notero is an ideal plugin for me!! thank for your great job! i think it's a way to sync the annotations as “Toggle Headings”. First, add the annotations as a “page” in the main body through API. Second, change the “page” to “Toggle Headings” in notion by users themselves. when there is a new update in zotero, just sync a new page and deleted the old toggle headings. it may be not a directly way, but i think it' s a way.

[mtillman14]

Hi, I am also adding my +1 to this idea! But, I wonder if it could be an option whether to do this syncing or not? I worry that having hundreds of annotated PDFs in a Notion page will drastically reduce the load speed. Is that a real concern?

[dvanoni]

@mtillman14 to be honest, I have no idea what the impact would be on Notion load speed, but I think it's a good point to bring up! Either way, I think making this feature configurable is a great idea. I imagine that some folks might not want to sync annotations as part of their workflow, so providing the flexibility to choose whether to enable this feature sounds smart to me.

[bvernot]

+1 to sync annotations! TBH, I thought that Notero already did that, and it was a big part of why I just installed it.

I liked @dvanoni 's structure for how to include annotations, at least conceptually. I have no idea about implementation issues.

One additional possibility: It would be nice if the color of the annotation was also preserved in the Notion page - this might help when visually scanning through them, so it's not just a wall of text?

Would image/area annotations also be synced?

[advaitathreya]

I like all of the ideas proposed here, and if I may add another suggestion for an option, I would like to embed the PDFs that I have stored in Zotero's storage in the Notion page so that I have my marked up PDF within the Notion database. It is possible to manually do this right now, so if Notero could have an option to add an embedded URL as a page block that would be awesome! Echoing everyone here, thank you for making this plugin!!

[dvanoni]

@advaitathreya thanks for the feedback!

I'd love to be able to automatically embed PDFs, but unfortunately it's currently not possible due to limitations with Notion. See the How to sync attached files into Notion section of the readme for more details.

[advaitathreya]

Looks like uploading local files automatically is not possible, but how about embedding PDFs of Zotero Storage files?

It looks like embedding PDF URLs is possible through the Notion API.

Retrieving the PDF view URL for the zotero item should be possible through the getBestAttachment method, and then modifying the URL to be of the form expected (Zotero file view URLs are of the form 'https://api.zotero.org/users/<UserID>/items/<ItemID>/file/view?key=<ZoteroAPIKey>').

I've been trying to do this and I think I have parts of it that are working, but I've never worked with JavaScript or APIs before, so I'm struggling with a few aspects. I will see if I can get a rudimentary working script to share, but maybe this would be faster for you? Or maybe I am completely off and it isn't possible, in which case apologies 😬

[josh-ashkinaze]

I think it would make sense that the notes sync can be configured to be either (1) Text in the body of a notion page or (2) An actual item field of the note called "Annotations"

(2) is convenient for short notes, but can affect load time. Anecdotally, from using Notion as a bibliography tool: If you put a large amount of text in the body of the page, this will not affect load time. However, a large amount of text as an item field will affect load time.

[dvanoni]

i think it's a way to sync the annotations as “Toggle Headings”.

In regards to the suggestion from @mai-m-ai above, the Notion team just added support for toggle headings to the API!

[eelshaikh]

i think it's a way to sync the annotations as “Toggle Headings”.

In regards to the suggestion from @mai-m-ai above, the Notion team just added support for toggle headings to the API!

How can this be used to transfer annotations?

[Prathyusha-konda]

Hi, thank you so much for the plugin, great functionality so far. Here to +1 the ideas on this thread. Definitely adding the feature to sync notes and annotations will be fantastic!! :)

[22-modifier]

@advaitathreya thanks for the feedback!

I'd love to be able to automatically embed PDFs, but unfortunately it's currently not possible due to limitations with Notion. See the How to sync attached files into Notion section of the readme for more details.

I wonder if the PDF itself truly has to be embedded... I feel like the goal is to have everything in the same place, minimizing the number of different apps. Now that Zotero has a more robust PDF reader/annotation tool (with associated iPad app), I wonder if the PDF truly needs to be embedded into Notion (particularly if the PDF annotations become tough to sync).

I tried to create a simple workaround that failed (just appending "/reader") to the Zotero URI in a notion table formula. Unfortunately Notion formulas can't output a URL. The web-based reader also doesn't allow annotations (yet). Would be nice if a Zotero URI link could just open the PDF in the desktop app.

[HameedahAhmad]

Hello, Thank you so much for the plugin, +1 for this ideas "sync annotation"

[a1576799]

I am looking forward to it. Currently, I have to take notes in zotero, export the markdown note, and paste it in Notion. However, the zetoro link is missing, so it cannot get back to zotero. Could you please also consider this in the future? THANKS!

[magdalenasabat]

+1 for "sync annotation"

[dr-ta-lia]

Hi, @dvanoni , Let me first tell you how much do I appreciate your work on zotero-notion bridge. I see that there have been quite quiet here in the past 3 months.. I understand that this have been more of a passion protect for you to help your partner with research. Notero has indeed become so much bigger! Thank you for great work and hope you keep it up! /+1 for notes sync from me ;)/

[do-not-perceive-me]

I like the idea of having the some short notes in the properties column.

I'm using a variation of the Advanced Database example, in the literature review view:

A possible idea for a workflow:

1. Define the names of your Notion database properties in Notero. For this example, it would be, Relevance, Problem or Purpose, etc.

2. (A) Notero creates a new note in Zotero with those properties as subheadings ( I would prefer this one but maybe it's harder to split the text and send it to Notion?)

(B) Notero creates separate notes for each of those properties

3. the text that you add under those notes gets pushed to the Notion database in that item property.

For longer text annotations:

I have no clue how to help with the zotero plug in, but I have some experience with the notion API and how to format pages so let me know if that's something that I could help with.

[Sayujr]

Absolute +1! This would complete the workflow! I think a simple text addition to the notion page under a defined heading like "Annotations" would suffice.

[jinghan23]

Hello, thank you so much for the plugin! Here to +1 for this ideas "sync annotation"

[machado-t]

+1

[VanillaEssence]

I'd also like the +1 this as this would be the most useful feature of Notero for me. It would be important that the Zotero links to pdf ie (example, 2017, p. 152) (pdf) were copied as links to Notion to. I currently have to Import from Markup (which is slow) > Copy text from new page to the page in the database. If this could be automated that would be amazing.

[fredericky123]

Absolute +1!

[dvanoni]

Hi everyone! I wanted to share a quick update to let you know I'm still working on this!

It turns out that converting HTML into Notion blocks is no small feat, but I've been able to make progress. I'm now able to sync most of the formatting supported by Zotero.

Once I have this in a state that's more usable, I'd love to know if anyone wants to try a test build to help me find any bugs. In the meantime, please enjoy this short video I recorded demonstrating various note formatting being synced into Notion.

Cheers!

https://user-images.githubusercontent.com/299357/232381123-40eb28a2-edcb-4354-9536-6b9efb1f1385.mp4

[atsyplenkov]

Hi @dvanoni,

Impressive progress! I'm ready to test and report any bugs.

[VanillaEssence]

I am super stoked for this, that looks great also happy to help with testing.

[SofiCourtney]

Thanks for all your amazing work, this will be a super helpful addition! I found this thread because I was looking for a similar functionality, that I'm not sure is included in this update (or maybe it is and i'm not catching that.) I'm working with the notero advanced database template. As I read through papers in Zotero, in addition to my own typed notes, I annotate the papers by highlighting a section of text and noting which column of the literature review tab they fit into (Relevance, Problem or Purpose, etc..). When I finish reading, I go through and cut and paste the highlighted text into the appropriate column in notion. I was wondering if it would be possible to automate this- my thought was to add a tag to the highlighted text in Zotero that specifies the column in notion where the text should be copied. I have no idea if something like this is complicated, or possibly you have already addressed it here?

[dvanoni]

Hey @SofiCourtney, thanks for the suggestion! I think this is a really interesting idea and something I'd like to explore.

I've pulled it out into its own issue (#256) because I do think it's a separate scope from this one. This issue is focused on syncing notes and annotations content into the body of the Notion database page so that it can support most of the formatting available in Zotero notes. Syncing content into specific database properties would work slightly differently because properties don't support as much formatting. I think we'd also need a way to specify which Zotero tags should map to which Notero properties so that folks can customize as needed.

[louaymd]

It’s finally (almost) here, thanks for your great work dvanoni! Happy to help with testing.

[dvanoni]

Hey everyone! I've just released version 0.4.8 which includes initial support for syncing notes!

I'd love to get feedback from y'all on how it works for you. Please see the release notes for details on how to use it.

Please note: this first release requires you to opt in to this new functionality by enabling it within Notero Preferences, and it does not sync notes automatically. Automatic syncing is planned for a future release.

[dmshapiro]

Thanks so much for working on this update (and this plugin overall, which is a lifesaver). One quick note from playing around with the experimental feature for syncing notes is that the highlight colors, even when turned on in Zotero, don't appear to sync over to Notion at the moment. I'm not sure if that's a feature that's yet to be released, but I saw it seemed to be supported in the release notes, so I thought it would be worth mentioning.

[dvanoni]

Hi @dmshapiro, thanks so much for testing this!

The current release does include support for syncing highlight colors, but it does have a limitation right now. It currently only syncs the highlight colors if they're exactly the same colors used by Zotero (defined here).

I believe this should work if you've performed highlighting within Zotero's built-in PDF reader. But, if you've highlighted in a different PDF tool, I think the colors can end up being slightly different than Zotero's colors.

Can I ask how you highlighted your PDFs?

[dmshapiro]

Ah, that makes total sense. I’m using what is almost certainly a different set of colors in Adobe Acrobat, using ZotFile to sync the files between Zotero and my Google Drive. If you’re able to include support for custom-defined colors that would be super helpful, but I understand that because Notion has a limited set of highlight colors that might be difficult to delineate between which colors should correspond with which in Notion. On Jul 30, 2023 at 5:31 PM -0400, David Hoff-Vanoni @.***>, wrote:

Hi @dmshapiro, thanks so much for testing this! The current release does include support for syncing highlight colors, but it does have a limitation right now. It currently only syncs the highlight colors if they're exactly the same colors used by Zotero (defined here). I believe this should work if you've performed highlighting within Zotero's built-in PDF reader. But, if you've highlighted in a different PDF tool, I think the colors can end up being slightly different than Zotero's colors. Can I ask how you highlighted your PDFs? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[dvanoni]

@dmshapiro, would you be willing to share one of your highlighted PDFs with me? I'd like to see an example of the colors coming from Acrobat. If you're up for it, feel free to share to my email: david@vanoni.dev

I'm hoping I can come up with a more flexible solution so that, if a color is "close enough" to one of the known highlight colors, then it will be correctly reflected in Notion.

Thanks!

[dmshapiro]

For sure, I’ll send that over. On Aug 2, 2023 at 2:21 AM -0400, David Hoff-Vanoni @.***>, wrote:

@dmshapiro, would you be willing to share one of your highlighted PDFs with me? I'd like to see an example of the colors coming from Acrobat. If you're up for it, feel free to share to my email: @. I'm hoping I can come up with a more flexible solution so that, if a color is "close enough" to one of the known highlight colors, then it will be correctly reflected in Notion. Thanks! — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.>

[asimukaye]

Hi, thanks for developing this brilliant plugin. I tried out the new annotations and notes integrations feature (thanks for adding that too). I was able to sync my notes but not the annotations. Am I missing something here? Also, the images in the notes didn't sync, but I am guessing that it is due to the lack of support for multimedia in the Notion API.

Edit: I figured that you can add all annotations as a note as a workaround and then sync the note.

[ThomasDeLange]

Hi, thanks for developing this syncing functionality! I am using it right now and it works very nice. The only feedback I have is that I would like to change toggle heading 1 to be toggle heading 3 and stay the same while syncing.

[dvanoni]

@asimukaye, I'm glad you found that workaround for extracting annotations to a note. Currently, that's the only method supported for syncing annotations. And you're correct about the limitation regarding images—that's currently not supported by the Notion API.

@ThomasDeLange, could you clarify which toggle heading you're referring to? In the below screenshot, are you referring to A or B? Perhaps I can make this a configurable option so that users can customize to their liking.

[dvanoni]

Hi @dmshapiro, I just released v0.4.10 which contains support for highlight colors other than Zotero's. Can you give it a try and let me know if it's working for you? Thanks!

[ThomasDeLange]

@dvanoni To make it a little complicated: I would love A to be dropdown heading 2 and B to be dropdown heading 3. This fits my notion page the best. Finally the text in B is also referenced again below. Would it be possible to remove that part?

But, this is all cosmetic and the functionality works great anyhow.

[menelic]

@dvanoni thanks for adding this great import of notes and annotations feature! In my tsting, it works flawlessly, it even imports notes made within the PDF, but in that case, the author of the note, which is part of the zotero note: is not synced. It would be great if this could be added, as it is important esp for collaborative work