laurent22 / joplin

Joplin - the privacy-focused note taking app with sync capabilities for Windows, macOS, Linux, Android and iOS.
https://joplinapp.org
Other
46.36k stars 5.04k forks source link

Having trouble syncing some images #1405

Closed xthursdayx closed 5 years ago

xthursdayx commented 5 years ago

Operating system

Application

After importing all of my Evernote notes, I've been slowly syncing my >80,000 items. However over the last couple of days I've been noticing some errors in the sync process. I get an orange bar at the top of Joplin that says "Some items cannot be synchronised. View them now". When I click this message I get the following information:

Items that cannot be synchronised

ad516503a11cd5ca435acc9bb6523536 (da21df1e87854081b5ac7e7deeb32481): File not found: /Users/wolfmac/.config/joplin-desktop/resources/da21df1e87854081b5ac7e7deeb32481.png ad516503a11cd5ca435acc9bb6523536 (51518c93f13947fdbb6ac9e352029692): File not found: /Users/wolfmac/.config/joplin-desktop/resources/51518c93f13947fdbb6ac9e352029692.png dde22b2b3d5347fc4ae8e651b38a9a34 (8b334101dbb546cbad5f0fa943e3e12f): File not found: /Users/wolfmac/.config/joplin-desktop/resources/8b334101dbb546cbad5f0fa943e3e12f.htm (d906d4419c07491986a567c05cf5b1db): File not found: /Users/wolfmac/.config/joplin-desktop/resources/d906d4419c07491986a567c05cf5b1db.jpg 9wbx (5daee42b04324de79dc6c8c665cdb47a): File not found: /Users/wolfmac/.config/joplin-desktop/resources/5daee42b04324de79dc6c8c665cdb47a.gif GetResizeImage (ea88447475a041048865f099d7582959): File not found: /Users/wolfmac/.config/joplin-desktop/resources/ea88447475a041048865f099d7582959 (aab0526076944acd8c68ceed89901934): File not found: /Users/wolfmac/.config/joplin-desktop/resources/aab0526076944acd8c68ceed89901934.jpg lyrs=m@171000000&hl=en&src=api&x=1146&y=1495&z=12&s=Galil (ccdb076ad3994673984790ae7afa2c37): File not found: /Users/wolfmac/.config/joplin-desktop/resources/ccdb076ad3994673984790ae7afa2c37.png ad516503a11cd5ca435acc9bb6523536 (fb61b93ab71b460e97c6113feb4ac0c0): File not found: /Users/wolfmac/.config/joplin-desktop/resources/fb61b93ab71b460e97c6113feb4ac0c0.png ad516503a11cd5ca435acc9bb6523536 (22a44511c73d4c4f9eaf6a1abfab7c37): File not found: /Users/wolfmac/.config/joplin-desktop/resources/22a44511c73d4c4f9eaf6a1abfab7c37.png lyrs=m@171000000&hl=en&src=api&x=1146&y=1495&z=12&s=Galil (517077002fe0489a8894643c87c69cb5): File not found: /Users/wolfmac/.config/joplin-desktop/resources/517077002fe0489a8894643c87c69cb5.png

These items will remain on the device but will not be uploaded to the sync target. In order to find these items, either search for the title or the ID (which is displayed in brackets above).

However, when I search my computer and within Joplin for the number sequences (both those in brackets and those not) my search returns nothing. I'm assuming these are large images. I have a number of very important research images in my notebooks, so I'd really like to make sure that they are recovered/attached to the correct notes. Any ideas?

laurent22 commented 5 years ago

On which device are you getting these errors? Is that the one where you've imported the evernote notes initially?

Did you get any error when you've imported the notes?

laurent22 commented 5 years ago

If you open the database.sqlite file you can find info about these resources. Just run the query select * from resources where id = "51518c93f13947fdbb6ac9e352029692". This for each missing resource. It might give you a name or something that will help you find what's missing.

From that you can locate the evernote note that's causing problem. Actually if you can find one and upload the enex of just that note here, it should be possible to fix that bug.

laurent22 commented 5 years ago

Actually I've just thought that maybe the search engine doesn't index resource ids anymore (i need to check), but you can manually search for it using sql again:

Select * from notes where body like "%51518c93f13947fdbb6ac9e352029692%"

And again that should tell you what evernote note is causing an issue.

xthursdayx commented 5 years ago

On which device are you getting these errors? Is that the one where you've imported the evernote notes initially?

Did you get any error when you've imported the notes?

Thanks for the help. In terms of isolating the cause: I'm seeing this error on my MacBook Air, the same device where I initially imported the Evernote notes (and where my Evernote database also resides), and there were no errors when importing the notes.

I just ran both queries you suggested, but for select * from resources where id = "51518c93f13947fdbb6ac9e352029692", I got the same information that was in the original error message: 51518c93f13947fdbb6ac9e352029692 ad516503a11cd5ca435acc9bb6523536 image/png, and for the query Select * from notes where body like "%51518c93f13947fdbb6ac9e352029692%" there was no result.

laurent22 commented 5 years ago

Could you try to search for some of these resource names in your Enex files, for example "9wbx", "GetResizeImage" or some other name and find the note that's causing problem? Then if you could export just that note (from Evernote), and re-import it into Joplin, we might find what the issue is.

From the look of it, the resource metadata was created but not the actual file, which is strange because if it's not created the import process should have stopped with an error.

xthursdayx commented 5 years ago

The problem is that I've (stupidly) already deleted some of my Enex files and some of the notebooks in Evernote since they appeared to be imported into Joplin without any issues. However, I was able to find one of the notes that has been causing the problems. I exported it again from Evernote and re-imported it into Joplin, but (like the first time) I there was no error or issue with the import. Both notes look the same in Joplin and have the same number of attached images...

laurent22 commented 5 years ago

Any chance you could upload that note here or send it directly to my email? https://raw.githubusercontent.com/laurent22/joplin/master/Assets/Adresse.png

I'm suspecting there might be a resource in there, maybe a hidden one, which is not imported properly and then causes these error messages.

xthursdayx commented 5 years ago

Sure, I've attached one of them here. Note.enex.zip

laurent22 commented 5 years ago

Thanks, the issue was that the Enex file contained an invalid resource with no data. It only has a "source url" which points to a gravatar image, so this type of resource will now be skipped. I expect this is what you saw for the other missing resource - probably small icons or images that were clipped from web content.

It's not clear to me why Evernote doesn't import these like other images, but it seems they can simply be skipped. Normal images and attachments you've manually added to notes should be imported fine, and all the main images from web clipper should be there too.

xthursdayx commented 5 years ago

Thanks a lot for checking into this @laurent22 - that’s a relief to me since I have over 80,000 notes, many of which are for work.

Is there a way to override the sync error messages in Joplin?

laurent22 commented 5 years ago

Is there a way to override the sync error messages in Joplin?

Currently not, other than manually deleting the resources on the sync target.