search

zadam / trilium

Build your personal knowledge base with Trilium Notes
GNU Affero General Public License v3.0
27.52k stars 1.88k forks source link

evernote import freezes #288

Closed ewingrj closed 5 years ago

ewingrj commented 5 years ago

I'm attempting to import my evernote contents. The .enex is ~800 mb. When importing, trilium freezes.

This is using macOS 10.14.2

zadam commented 5 years ago

Hmm, that's an issue - I've never been able to managed such large ENEX imports since I don't have any file of such size.

Could you post log from your data directory? It's stored in /Users/[user]/Library/Preferences/trilium-data/logs

ewingrj commented 5 years ago

there is no directory /Users/[user]/Library/Preferences/trilium-data/.

Also, it appears something is killing the server, not necessarily the size. I was able to import 1 notebook ~ 10mb file, but another notebook of the same size fails and seems to kill the server

zadam commented 5 years ago

I'm sorry, it's actually /Users/[user]/Library/Application Support/trilium-data/log

Interesting, it is very possible that it's not size related. Logs would probably help in that case.

ewingrj commented 5 years ago

okay, I just cloned the repo & got it running...

the error is:

{ Error: Unsupported MIME type: image/x-icon
    at Jimp.throwError (/Users/rjewing/code/external/trilium/node_modules/@jimp/utils/dist/index.js:26:13)
    at Jimp.parseBitmap (/Users/rjewing/code/external/trilium/node_modules/@jimp/core/dist/utils/image-bitmap.js:119:32)
    at Jimp.parseBitmap (/Users/rjewing/code/external/trilium/node_modules/@jimp/core/dist/index.js:498:32)
    at new Jimp (/Users/rjewing/code/external/trilium/node_modules/@jimp/core/dist/index.js:451:13)
    at /Users/rjewing/code/external/trilium/node_modules/@jimp/core/dist/index.js:972:5
    at new Promise (<anonymous>)
    at Function.Jimp.read (/Users/rjewing/code/external/trilium/node_modules/@jimp/core/dist/index.js:971:10)
    at resize (/Users/rjewing/code/external/trilium/src/services/image.js:47:30)
    at Object.saveImage (/Users/rjewing/code/external/trilium/src/services/image.js:15:32)
    at saveNote (/Users/rjewing/code/external/trilium/src/services/import/enex.js:228:52) methodName: 'constructor' }
{ Error: Unsupported MIME type: image/x-icon
    at Jimp.throwError (/Users/rjewing/code/external/trilium/node_modules/@jimp/utils/dist/index.js:26:13)
    at Jimp.parseBitmap (/Users/rjewing/code/external/trilium/node_modules/@jimp/core/dist/utils/image-bitmap.js:119:32)
    at Jimp.parseBitmap (/Users/rjewing/code/external/trilium/node_modules/@jimp/core/dist/index.js:498:32)
    at new Jimp (/Users/rjewing/code/external/trilium/node_modules/@jimp/core/dist/index.js:451:13)
    at /Users/rjewing/code/external/trilium/node_modules/@jimp/core/dist/index.js:972:5
    at new Promise (<anonymous>)
    at Function.Jimp.read (/Users/rjewing/code/external/trilium/node_modules/@jimp/core/dist/index.js:971:10)
    at resize (/Users/rjewing/code/external/trilium/src/services/image.js:47:30)
    at Object.saveImage (/Users/rjewing/code/external/trilium/src/services/image.js:15:32)
    at saveNote (/Users/rjewing/code/external/trilium/src/services/import/enex.js:228:52) methodName: 'constructor' }
zadam commented 5 years ago

Thanks! It looks like it fails on unrecognized image type (x-icon). This should be easy to fix.

zadam commented 5 years ago

This will be fixed in the next release. Unrecognized image types (like icon) will be saved as a file note.

ewingrj commented 5 years ago

unfortunately this fix doesn't work and the same error still happens. It looks like this is throw when reading the img buffer.

Is the buffer a base64 encoded image and the mimetype is set differently then what the export file contains?

Here is 1 of the resources that was causing an error, not sure if that helps or not. Otherwise I can get you an .enex file to reproduce with.

{ title: 'favicon.ico.png',
  attributes:
   [ { type: 'label', name: 'fileSize', value: 1148 },
     { type: 'label', name: 'originalFileName', value: 'image.png' },
     { type: 'label',
       name: 'sourceUrl',
       value:
        'http://www.apieceofrainbow.com/wp-content/themes/foodiepro/images/favicon.ico' },
     { type: 'label',
       name: 'originalFileName',
       value: 'favicon.ico.png' } ],
  content:
   <Buffer 00 00 01 00 01 00 10 10 00 00 00 00 20 00 68 04 00 00 16 00 00 00 28 00 00 00 10 00 00 00 20 00 00 00 01 00 20 00 00 00 00 00 00 08 00 00 00 00 00 00 ... >,
  mime: 'image/png' }
zadam commented 5 years ago

Is the buffer a base64 encoded image and the mimetype is set differently then what the export file contains?

That's possible. Here it says the image is image/png, but then the library fails on not supporting image/x-icon mime type.

ewingrj commented 5 years ago

so it appears that x-icon is the standard mime type for favicon.ico. I created an issue for jimp to support that mime type.

In the mean time, I've made a PR #312 that will check the mime type from the buffer, as well as better handle img errors

nhendin commented 4 years ago

I am seeing some similar issues --- only some enex files import and most seem to hang Trilium.

I can attach one that fails. I am running Trilium on docker on ubuntu 18.04 LTS, and see time out errors, as well as some errors in the logs seeming to indicate missing libraries in the container.

trilium-2020-06-27.log.gz docker-container.log.gz

Evernote files: fails.zip

Thanks.