FR - Import annotations

[bepolymathe]

Hi @petervelosy,

Thanks for this great project ! This is already very useful !

Since version 6, this software is not only an excellent free bibliography management software but also a great annotation and note taking tool.

I would find it amazing to be able to import the annotations of a document into freeplane to organize them and think about the structure of a project (With the same philosophy, i.e. everything stays in Zotero). For my part, I will even start writing in Freeplane (I already do) thanks to the excellent MarkdownHelper addon

It must be technically possible, especially since there are links to the annotations.

This would allow us to regain the mindmap/writing fluidity that we started to have a few years ago with the Docear project. In my work as a researcher it would be wonderful and I imagine it would be of interest to students and in general to anyone who needs visual thinking and documentation.

[petervelosy]

This is an amazing suggestion.

Technically it is possible, however, it will require freeplane-zotero to communicate with Zotero on a secondary channel too. It currently uses the HTTP citation API, which also serves as the backend for word processor integrations: https://www.zotero.org/support/dev/client_coding/http_integration_protocol . However, this only contains the functions required for generating citations.

This secondary channel could be directly reading Zotero's sqlite database: https://www.zotero.org/support/dev/client_coding/direct_sqlite_database_access . Here, it is possible to lookup the work cited by using the item_id in the CSL_CITATION object that gets saved as an attribute of the Freeplane node.

I am thinking about the problem of change tracking, however. I imagine in the initial brainstorming stage, it would be nice to just be able to sync a node with Zotero so that all of the document's annotations get updated as child nodes of the node the document is cited at. However, you might later want to 1) move the annotations around, possibly to a subtree of the original node, and 2) edit the annotation so that it becomes a fully elaborated concept.

My current idea is to have a Refresh Annotations function(to work on one or more selected nodes or the whole mindmap), that would update all nodes corresponding to imported annotations, even if they have been moved into a subtree. (Actually, you could even keep the relationship alive with custom attributes such that it lets you completely detach the annotation from the original node and move it elsewhere). An imported annotation would, however, be a special node that can always be overwritten by executing Refresh Annotations.

Some further operations on annotation nodes could be:

• Detach from Zotero: disconnect imported annotation nodes from Zotero such that they wouldn't get overwritten anymore (i.e. the user would be free to edit their text).
• Delete and Ignore ("Throw away"): Delete an imported annotation node and add some metadata with the ID of this annotation to the mind map such that it does not get reimported again. Btw, Detach would also move the annotation the detached node was based upon into this ignore list, so that it does not reappear.

I am not sure we need Detach from Zotero, though. An alternative philosophy would be that you either keep an annotation as a part of your map forever (thus keep refreshing it), or you eventually throw it away (Delete and Ignore) because by that time you will have built all the contents of the annotation in to other nodes. When you build concepts using annotations, you will most probably use multiple annotations for forming a new concept, so the case where you would directly edit an annotation in order to turn it to something more elaborate (and thereby detach it from Zotero) would probably be an edge case.

[bepolymathe]

Thank you for these rich reflections in which I find myself in large part. This project is very exciting 😃

Reading the sqlite database is the best solution and plugins for obsidian or logseq also use this solution I think because it is faster.

Some comments for the rest:

• Editing annotations would be great but the most important is moving nodes in the map without changing the link to the original.
• Another question is the use of the same annotation in several places of the map (clone in freeplane?) or in different maps.
• In case of import, one could probably use the filters of zotero... import only the yellow annotations for example.

In the case, more and more frequent, of someone who write in markdown and then export with pandoc it is interesting to consider betterbilatex and the citation keys which ensure a form of perennity to the source project.

Here are some reactions as I am in travel. Available to discuss, test, translate, support 😉. I could make you maps to explain my ideas from Tuesday...

[bepolymathe]

Hi @petervelosy

Just by adding a few things that come to mind, it is important to think about the format of the annotation import as well to allow integration with freeplane.

Freeplane and Zotero.zip

[petervelosy]

I've had a look at your three options, here are some of my thoughts:

I am actually tending towards a fourth option, that is, to have the highlighted text and the note imported as a parent node having the highlighted text and a child node having the note. The idea behind is that I'd prefer everything to be visible at once after import. (As far as I am aware, the function View > Display notes on map in Freeplane is all-or-nothing, so there is no way to turn this on for selected nodes only. If, after import, you'd have to open all existing Freeplane notes to see everything you have imported, your map view might get very cluttered).

Furthermore, certain notes won't make sense without their context (e.g. when a note starts with "This means that..."). Also, I think the hierarchy between highlighted text segments and notes is something that should be kept very flexible. When writing a review article, you might want to keep the main (highlighted) theses of the one or the other article as the structural cornerstones, which you explain by the means of the notes, whereas in other cases, the notes could contain higher level concepts that you use for building an article and the highlighted text is a detail reference.

What do you think?

P.s. I had a random idea that the way Annotations are being imported could even be organized according to the chapters of the original PDF document (if the original PDF has headings defined), by matching up Zotero's annotation location coordinates with the heading locations (and widths - to determine how many columns the document has :-) ). But I'll leave that to the time when the project (hopefully) gets more contributors and there is someone who can dive into PDF parsing. As of now, imported annotations could only be ordered by page number, which would at least give the import some sort of a structure.

P.s. 2. A complete workflow with Zotero -> Freeplane -> Document processor would be amazing to achieve. However, I personally mostly write in LaTeX directly, so I don't know Pandoc's extended Markdown format well. Getting to know it better would help resolve the fundamental issue of whether there should be a one-way relationship between the FP map and the document (that is, you really write everything in Freeplane, MarkdownHelper-style, and you never touch the document manually), or it should rather be a soft semantic link (i.e. the resulting document can be edited, but it has some special tags through which it retains references to parts of the FP map). A great way to decide this would be to build a proof-of-concept of a Freeplane map that depicts a scientific article in it's entire complexity (with tables, images, cross-references, maths formulae, - and conceptual map nodes not to be compiled into the article - etc.). Would you perhaps be available to build such a proof-of-concept, to see, how comfortable it would be to work with?

[bepolymathe]

Hi @petervelosy

I am actually tending towards a fourth option, that is, to have the highlighted text and the note imported as a parent node having the highlighted text and a child node having the note. The idea behind is that I'd prefer everything to be visible at once after import. (As far as I am aware, the function View > Display notes on map in Freeplane is all-or-nothing, so there is no way to turn this on for selected nodes only. If, after import, you'd have to open all existing Freeplane notes to see everything you have imported, your map view might get very cluttered).

On the general principle I agree with you. I thought with the annotation text in the perspective of a finer writing work with MarkdownHelper... but it touches on your proposal of end of message (I'll come back to it).

I had a random idea that the way Annotations are being imported could even be organized according to the chapters of the original PDF document (if the original PDF has headings defined), by matching up Zotero's annotation location coordinates with the heading locations (and widths - to determine how many columns the document has :-) ). But I'll leave that to the time when the project (hopefully) gets more contributors and there is someone who can dive into PDF parsing.

This is a good idea that could be useful and save time for taking reading notes directly in Freeplane. That said, I am modifying the structure of my notes underneath the article to take into account the questions I am trying to answer. 🤔

As of now, imported annotations could only be ordered by page number, which would at least give the import some sort of a structure.

Yes, it looks good to me, especially since we always need to quote the page of an extract in an article.

Getting to know it better would help resolve the fundamental issue of whether there should be a one-way relationship between the FP map and the document (that is, you really write everything in Freeplane, MarkdownHelper-style, and you never touch the document manually), or it should rather be a soft semantic link (i.e. the resulting document can be edited, but it has some special tags through which it retains references to parts of the FP map). A great way to decide this would be to build a proof-of-concept of a Freeplane map that depicts a scientific article in it's entire complexity (with tables, images, cross-references, maths formulae, - and conceptual map nodes not to be compiled into the article - etc.). Would you perhaps be available to build such a proof-of-concept, to see, how comfortable it would be to work with?

I think this is a very good way to progress. I'll try to make a sample card with lorem ipsum that explains a little bit how I see the whole thing... tomorrow probably.

PS: I thought it would be interesting to put EdoFro (author of MarkdownHelper) in a discussion at some point. See when.

[bepolymathe]

Hi @petervelosy

I played with the idea a bit. I'm sharing with you what I have now because I'm going to have little time this afternoon and tomorrow. I will continue to feed this card for testing.

POC Zotero-Freeplane-Document.zip

You need to install MarkdownHelper to import styles and see "document"

[petervelosy]

Hi @bepolymathe!

Thanks a lot for the PoC! I will explore your ideas in depth and also do a practical experiment myself, since I am going to put together some PhD project proposals in the coming days/weeks.

In the meantime I 've done another PoC of performing the technical annotation import itself. It works well by accessing the sqlite database. I think we should not go too fast here and rather develop a mature concept that would that be implemented in the next freeplane-zotero release. Changing the concept later, when other people will already have produced content with this function (so you need to consider backwards compatibility...) is always harder.

Nevertheless, if you want to experiment with what is already there, I can push the latest code to a branch and help you set up the build process.

My first approach would be to have a strongly opinionated workflow (i.e. to find one way to import annotations that covers a large enough range of use cases) instead of making the resulting data structure configurable (i.e. you'd have configuration options to switch between various representations of an annotation (a highlighted text segment) and its comment), as the latter would increase complexity in a way that slows down future development considerably. Of course, if having one representation turns out to be impracticable, a configurable representation is always a possible escape path.

A discussion with EdoFro about such a workflow would definitely be interesting. Are you already in contact with him regarding MarkdownHelper?

[bepolymathe]

A discussion with EdoFro about such a workflow would definitely be interesting. Are you already in contact with him regarding MarkdownHelper?

Yes 18 minutes ago ;-)

Edit (2023-02-08T10:17): and he replied @petervelosy

https://github.com/EdoFro/Freeplane_MarkdownHelper/discussions/13

[bepolymathe]

In the meantime I 've done another PoC of performing the technical annotation import itself. It works well by accessing the sqlite database. I think we should not go too fast here and rather develop a mature concept that would that be implemented in the next freeplane-zotero release. Changing the concept later, when other people will already have produced content with this function (so you need to consider backwards compatibility...) is always harder.

Totally agree with this.

Nevertheless, if you want to experiment with what is already there, I can push the latest code to a branch and help you set up the build process.

Why not

My first approach would be to have a strongly opinionated workflow (i.e. to find one way to import annotations that covers a large enough range of use cases) instead of making the resulting data structure configurable (i.e. you'd have configuration options to switch between various representations of an annotation (a highlighted text segment) and its comment), as the latter would increase complexity in a way that slows down future development considerably. Of course, if having one representation turns out to be impracticable, a configurable representation is always a possible escape path.

Ok with that too 👍🏼

Thank you for your efforts.

PS: What is your PhD about?

[bepolymathe]

Just for documentation here --» http://reagle.org/joseph/2009/01/thunderdell.html ;-)

[petervelosy]

Just for documentation here --» http://reagle.org/joseph/2009/01/thunderdell.html ;-)

Thank you, that's a very interesting project.

P.s. I have created an initial PoC implementation of importing annotations, you can try it out by building this branch: https://github.com/petervelosy/freeplane-zotero/tree/feature/7_import_zotero_annotations

It can already import highlighted text segments (along with their page numbers) and notes, if any (Tools/Zotero/Import annotations). It also registers an init script with a listener for node deletion events, this automatically offers the possibility to put deleted annotations to an ignore list (so that they wouldn't get reimported).

As Freeplane init scripts are formally not supported in the addon framework yet, Freeplane will consider it not as a part of a trusted addon but as an untrusted user script. Thus, you might get a security warning about executing a potentially harmful script. As this is suboptimal from a UX point of view, it should be clarified whether there is any alternative way for an addon to register listeners at startup or whether init scripts can be integrated into the addon framework.

Please note that this initial implementation is unstable and untested, I don't recommend using it on any production mind maps yet.

[petervelosy]

P.s. as for my PhD: I am not a PhD student yet, I'm currently writing up some proposals for potential supervisors in order to start a PhD. Let's keep this GitHub issue to the point, but I'm happy to discuss more about that, too, if you are interested, you can drop me a mail at peter@[githubusername].com

[petervelosy]

For reference: here's my forum post in the Freeplane forums for solving the security warning problem with the init script registering the node deletion listener: https://github.com/freeplane/freeplane/discussions/1041

[EdoFro]

For reference: here's my forum post in the Freeplane forums for solving the security warning problem with the init script registering the node deletion listener: freeplane/freeplane#1041

also for reference, FreeplaneGTD made a little change to solve this problem. here: https://github.com/gpapp/FreePlaneGTD/pull/127/commits/f7a497f7d183530ecbe13a24f6dd96c289cdbfee

[petervelosy]

For reference: here's my forum post in the Freeplane forums for solving the security warning problem with the init script registering the node deletion listener: freeplane/freeplane#1041

also for reference, FreeplaneGTD made a little change to solve this problem. here: gpapp/FreePlaneGTD@f7a497f

From what I see, this makes sure the listener only gets registered when the addon is active. This certainly makes sense to implement in freeplane-zotero as well. My problem is, however, this: under the default Freeplane settings (Script execution enabled = Ask), the following dialog is triggered when the init script loads:

Such a dialog is not triggered when executing any scripts coming from an addon, as these are considered trustworthy by default. However, the init script is not regarded as such, even if it is installed by the addon, as it gets installed to /scripts/init instead of /addons/freeplane-zotero/scripts An alternative would be for me to sign the init script and let the user import my public key into their default Java keystore before installing freeplane-zotero, but that would be too burdensome from a UX perspective, unless it can be automated as a part of the addon installation process.

[bepolymathe]

Hi @petervelosy

Sorry for my absence, I was taken by a lot of unexpected things. I want to try and give feedback if you already have a first version of the addon compiled. If not, I'm available when you need me.

[bepolymathe]

Hi @petervelosy

Just for documentation, there is a contributor ( @jahagirdar ) who try to import annotations from PDF file via Python

https://github.com/freeplane/freeplane/discussions/952#discussioncomment-4727008