search

Xunius / Menotexport

Python solution to export annotations from your Mendeley library.
GNU General Public License v3.0
124 stars 20 forks source link

List index out of range Menotexport v1.4 Gui WIndows #10

Closed supercourgette closed 7 years ago

supercourgette commented 7 years ago

Thanks for your work!

I was hoping to export some folders of Mendeley. However when using the pre build binary GUI of windows I got the following error after entering my "Mendeley Data File".

# <Menotexport>: Failed to recoganize the given database file.
list index out of range

The data file is correct (at the default location) and I updated sqlite3.dll as advised, but the error remains.

PS : It is recognize not recoganize. ;)

Xunius commented 7 years ago

Hi supercourgette, I couldn't reproduce your error. It seems that it failed to fetch a list of folders in your mendeley library. Could you give me some info regarding your folders, like do you have nested folders (folders inside folders), or does any one of them has some special characters in their names, including spaces. Sorry for the trouble.

supercourgette commented 7 years ago

Yes, I have about 50 folders in total. Some are subfolders and some are even subsubfolders. And yes, there is some special characters in the name, like spaces, ( ), [ ], and -. I used what Mendeley allowed me to use...

Example below :

-Research proposal Weather-Buildings --State of the Art ---Simulation Weather ---Rendering Weather [urban level] -Deformation Buildings (facades)

No worries, I hope you can solve it.

Xunius commented 7 years ago

Sorry I'm still quite lost. I'll try looking deep later this week.

Xunius commented 7 years ago

Hi supercourgette, Sorry I didn't figure out what's going wrong. I tried creating folders with every symbol I could find on my keyboard, nesting them multiple levels, and spamming 50 of them, still I cant reproduce your error. Regarding the windows sqlite.dll, I tried replacing it with the 64-bit DLL (x64) for SQLite version 3.15.1 (I assume this is the one you obtained), again works no problem, and I don't think the specific version matters.

Do you observe anything else in your library that could possibly mess things up? Does your folder contain non-english characters? I tried inserting some chinese characters and it worked OK though.... Other than that, I currently have no idea what could be wrong. Do you have any experience with python and I wonder whether you could set a break point and step into the code and help me with some diagnoses?

Xunius commented 7 years ago

Hi. I've made some minor changes and created a new copy for win10. Could you please download from https://dl.dropboxusercontent.com/u/74222489/menotexport-gui-win10-64bit-debug.rar, and give it a try see it gets anywhere?

supercourgette commented 7 years ago

Hi, thanks for your reply. Unfortunately, the version you provided give me a similar error:

     Problem retrieving folder name for id 0
# <Menotexport>: Failed to recoganize the given database file.
list index out of range

Assuming the folder name for id 0 is the first folder I created in my list, I can give you its name:

Generalized Cylinder

No nested folder inside

Among the possible issues I can think of:

That's pretty much it.... :(

I can check later in the weekend yes.

Xunius commented 7 years ago

Hi, Thanks for the feedbacks.

Sorry I probably overlooked one issue in the previous version. I meant to let the programme put an "unknown" as a placeholder when it fails to retrieve the folder name so that it could proceed along. Then in the folders dropdown menu you could see 'unknown' and many others, from there you should be able to select those recognized ones to process, and don't select "All" or "unknown" from the list. I've made one little change, could you give it one more try: https://dl.dropboxusercontent.com/u/74222489/menotexport-gui-win10-64-debug2.rar ?

If you also use firefox, there is an add-on called "SQLite Manager" that allows you browse into the sqlite data file. I actually use that tool to help developing this software. If you could get that installed, use it to open the mendeley data base file, and select "Folders" on the left-hand column, you could see which folder is causing the issues (that with an id 0). Mine id starts from 1, so that could be the reason. Also I didn't have any groups previously. I tried creating one with some folders in the group, and the tool fetches those properly.

If this one fails again, I wonder do you mind sending a copy of your mendeley sqlite file to me so I could have a closer look? You could send to my email if you feel comfortable with that: xugzhi1987@gmail.com.

supercourgette commented 7 years ago

Good news, even if it is not working yet, there are some advancements.

     Error3: Problem retrieving folder name for id 0

     Error2: Problem retrieving parent folder id for id 1

     Error3: Problem retrieving folder name for id 0

     Error2: Problem retrieving parent folder id for id 3

     Error3: Problem retrieving folder name for id 0

     Error2: Problem retrieving parent folder id for id 4

     Error3: Problem retrieving folder name for id 0

     Error2: Problem retrieving parent folder id for id 5

     Error3: Problem retrieving folder name for id 0

(etc)

But after all those errors, Menotexport manage to retrieve some folders (some with the name "unknow", some with their correct name). Yet, it fail to do any actions.

Moreover, thanks to you I think I know how you can reproduce a part of the error. In the SQLITE some of the folders rising an error in Menotexport have a parent ID of 0, yet I do not have a folder with an ID of 0 (the first ID is 1).

When created at the root, a folder (eg. folder X) has a parent ID of -1. In this case, Menotexport can retrieve it without raising an error. But. If you move a folder under another (eg. move folder X under a folder Y), then move this folder at the root (eg. move folder X at the root); the folder will have a parent ID of 0 (eg. folder X has a parent ID of 0). In this case, Menotexport raises the following error:

     Error3: Problem retrieving folder name for id 0

     Error2: Problem retrieving parent folder id for id 1

Maybe this is it? In any case, I sent you my mendeley sqlite on your gmail address.

Xunius commented 7 years ago

Hi, Thanks so much it is really helpful discovery. It might take me ages before I notice this weird behavior if you didnt' tell me. Now it is fairly easy to fix: it simply needs to recognize both -1 and 0 as the parent id for root folders.

Please fetch the updated version from dropbox again and check it out. And if you encounter any other issue just post here.

P.S. another line of warning though: currently if your folder contains "/" in their names, it won't work properly (it reads that as folder hierarchy rather than literal folder names). I will patch this bug later.

supercourgette commented 7 years ago

Hi, I just tried and it is now working as intended (I didn't do a thoughtful check, but the output appears to be correct). Thanks a lot for your reactivity. :)