Error when extracting notes for some papers in Mac

[marieke-woensdregt]

Hi Xunius,

Thank you very much for creating this tool! Very handy, especially now that Mendeley has started encrypting its local database.

I'm trying to run Menotexport on Mac, but it doesn't recognize my local database file for some reason. I installed all the dependencies you listed and installed Menotexport from the source code.

I'm running the following command: python menotexport.py -pmns /Users/pplsuser/Library/Application\ Support/Mendeley\ Desktop/<my email>@www.mendeley.com.sqlite /Users/<my username>/Documents/papers_annotated

And it does start running, and runs correctly for a bunch of papers, but at some point it hits a paper where it gives the following error: # <Menotexport>: Retrieving notes... Traceback (most recent call last): File "menotexport.py", line 1486, in <module> args.separate,args.zotero,args.verbose) File "menotexport.py", line 1324, in main fidii,fnameii,allfolders,action,separate,iszotero,verbose) File "menotexport.py", line 1077, in processFolder annotations,flist=extractAnnos(annotations,action,verbose) File "menotexport.py", line 996, in extractAnnos nttexts=extractnt.attachRefTextsToNotes(nttexts,hltexts) File "/Users/pplsuser/Downloads/Menotexport-master/lib/extractnt.py", line 123, in attachRefTextsToNotes if isInside(x0,y0,hljj.bbox): File "/Users/pplsuser/Downloads/Menotexport-master/lib/extractnt.py", line 79, in isInside x1,y1,x2,y2=rect TypeError: 'NoneType' object is not iterable

(Apologies for the messed-up formatting.)

If I remove that particular paper from my Mendeley Desktop and run it again it gets a bit further, but then it hits another paper where it gives the same error..

Thanks, Marieke

[Xunius]

Hi marieke-woensdregt,

Can you let me know how you installed the dependencies and Menotexport? And what's the version of PyPDF2 and pdfminer, in case those modules behave differently in a newer version. Lastly, if you don't mind, can you send me copy of the PDF that triggered that error, and let me know the location where you made some highlights and a nearby sticky note (maybe take a screenshot)? Sorry for the trouble.

[marieke-woensdregt]

Hi Xunius,

Thanks for your quick response! Strangely enough I just ran it without the [-n] option selected (python menotexport.py -pmbz <dbfile> <outputdir>) and that ran perfectly fine without giving any errors, and it has miraculously extracted all the sticky notes and general notes (from the right-hand sidebar in Mendeley) as well (I thought it would only do that if I selected the [-n] option). So I already have what I wanted now. But nevertheless, here are the details you requested:

How I installed the dependencies + versions of pypdf2 and pdfminer:

• conda install -c conda-forge pypdf2 ---> Version: 1.26.0
• conda install -c anaconda pandas
• conda install -c conda-forge pdfminer ---> Version: 20140328
• conda update numpy (already installed)
• conda install -c anaconda beautifulsoup4
• sqlite3 I installed by following the instructions outlined here for Mac OS X. So what I did is:
  1. downloaded sqlite-autoconf-3240000.tar.gz from here. And then from the command line, ran:
  2. tar xvfz sqlite-autoconf-3240000.tar.gz
  3. cd sqlite-autoconf-3240000
  4. ./configure --prefix=/usr/local
  5. make
  6. make install
• poppler I installed using Homebrew: brew install poppler

How I installed Menotexport:

1. Downloaded the zip
2. Unzipped
3. cd Menotexport-master/
4. python setup.py build
5. python setup.py install

Attachments: I've attached one of the pdfs that triggered the error (not the original one, because I removed that from my Mendeley to see if maybe it happened to be just that single document that was problematic), and two screenshots of a place in that pdf where I made some highlights and a nearby sticky note. Keysar_etal_2000_TAKING PERSPECTIVE IN CONVERSATION- The Role of Mutual Knowledge in Comprehension.pdf

Thanks! Marieke

P.S. I also tried using Menextract2pdf, but that is giving me yet another error (something to do with a missing EOF marker in a pdf of a book chapter). And I prefer Menotexport because it maintains the folder structure.

[marieke-woensdregt]

P.P.S. Attached here a pdf of the paper that I got by running only python menotexport.py -pmbz <dbfile> <outputdir> so without the [-n] function selected. Keysar et al. - 2000 - Taking perspective in conversation the role of mutual knowledge in comprehension(3).pdf

[Xunius]

Thanks a lot for taking the time to provide all those info. I think I've found the issue. Partly because of some mistakes in the code, which I've made a change, partly due to something in your system: I guess the software pdftotext, which should be installed in poppler, is not working, or for some reason Menotexport failed to acknowledge its existence. Anyway, the fixed version should remove that error whether pdftotext or pdfminer is used.

There is perhaps some ambiguity in the doc, but the -n flag was meant to output notes into a txt file. Your -pmbz flag worked because the -p flag, which exports the PDFs files, implies retaining those highlights and notes in the output PDFs, and there won't be a separate txt file in this case.

[marieke-woensdregt]

Ah I see, so I misunderstood the readme file. I just ran the newer version just to check and it all works perfectly. Thank you for addressing the issue so quickly!