ciur / papermerge

Open Source Document Management System for Digital Archives (Scanned Documents)
https://papermerge.com
Apache License 2.0
2.55k stars 267 forks source link

Problem with IMAP import #404

Open frdmn opened 3 years ago

frdmn commented 3 years ago

Description

I have a little issue with the IMAP import, that should pull documents from my self-hosted mail server but doesn't seem to function at the moment.

Expected

Papermerge should pull and import documents attached by mail via IMAP.

Actual

Papermerge connects to the inbox, sees unread mails, but doesn't actually import them into the system.

Info

papermerge.conf.py configuration file

#DBTYPE = "mariadb"
DBUSER = "root"
DBPASS = "root"
DBHOST = "mariadb"
DBNAME = "papermerge"

MEDIA_DIR = "/data/media"
STATIC_DIR = "/app/papermerge/static"
MEDIA_URL = "/media/"
STATIC_URL = "/static/"

BINARY_STAPLER = "/usr/local/bin/stapler"

IMPORTER_DIR = "/importer"

IMPORT_MAIL_HOST = "obi-wan.yeahwh.at"
IMPORT_MAIL_USER = "docs-import@mailbox.tld"
IMPORT_MAIL_PASS = "pw"
IMPORT_MAIL_DELETE = True
IMPORT_MAIL_BY_USER = True

OCR_DEFAULT_LANGUAGE = "eng"

OCR_LANGUAGES = {
    "eng": "English",
    "deu": "Deutsch"
}

Relevant logs

[2021-07-12 17:19:50,281: INFO/Beat] Scheduler: Sending due task import_from_local_folder (papermerge.core.management.commands.worker.import_from_local_folder)
[2021-07-12 17:19:50,283: INFO/Beat] Scheduler: Sending due task import_from_email (papermerge.core.management.commands.worker.import_from_email)
[2021-07-12 17:19:50,283: INFO/MainProcess] Task papermerge.core.management.commands.worker.import_from_local_folder[0ca51d89-2e89-4537-af31-61f9d595f1d9] received
[2021-07-12 17:19:50,284: INFO/MainProcess] Task papermerge.core.management.commands.worker.import_from_email[c96275ec-3939-4eb8-bdd0-26d3b55df9d8] received
[2021-07-12 17:19:50,284: DEBUG/ForkPoolWorker-2] Celery beat: import_from_local_folder
[2021-07-12 17:19:50,285: INFO/ForkPoolWorker-2] Task papermerge.core.management.commands.worker.import_from_local_folder[0ca51d89-2e89-4537-af31-61f9d595f1d9] succeeded in 0.0008253850974142551s: None
[2021-07-12 17:19:50,286: DEBUG/ForkPoolWorker-2] Celery beat: import_from_email
[2021-07-12 17:19:50,605: DEBUG/ForkPoolWorker-2] IMAP Import: UNSEEN messages 1 count
[2021-07-12 17:19:50,644: INFO/ForkPoolWorker-2] Task papermerge.core.management.commands.worker.import_from_email[c96275ec-3939-4eb8-bdd0-26d3b55df9d8] succeeded in 0.3582966411486268s: None
[2021-07-12 17:20:19,569: INFO/Beat] Scheduler: Sending due task rebuild_the_tree (papermerge.core.management.commands.worker.rebuild_the_tree)
[2021-07-12 17:20:19,570: INFO/MainProcess] Task papermerge.core.management.commands.worker.rebuild_the_tree[a0a717a1-ab08-4c3f-a1a3-cce5d649d462] received

Is there anyway to further troubleshoot this problem?

Many thanks for working and developing on Papermerge - I love the product! ❤️

ciur commented 3 years ago

There may be a problem with document itself i.e. application fails to OCR and/or read data about imported document.

What document format is it ? Is it a PDF file ? TIFF ? Jpeg? One very simple way to double check that document is "importable" i.e. there are no errors while importing the document, aside from IMAP part, is to upload document from email attachment into the application via web interface.

frdmn commented 3 years ago

I've tried with a bunch of random (mechanically generated) PDFs which otherwise worked out fine when importing those manually. None of the few (about 10 different documents) were imported nor were there any other relevant log messages unfortunately.

schnickalodeon commented 3 years ago

Hey there,

Unfortunately I have the same issue with version 2.0

image

The Server sees the unseen messages but don't import them. If I try to upload the document manually it works.

Can I help somehow to solve the Issue?

Many Thanks for your work 🙏

b0wtie commented 2 years ago

I had the same problem. It was - in my case - caused by the way, the Apple Mail program attaches files to Emails.

When I sent an email with the attachment via the Apple Mail program it did not work. When I used a webmailer it did.

t-lie commented 2 years ago

I have also an issue with IMAP import and latest linuxserver docker image. It will always import all new mails/documents within admin account, even the sender mails fits to an other user account and the "routing by user" is activates within config file, and within settings of admin and other user.

schnickalodeon commented 2 years ago

Yes, that was the issue - Thanks a log :)

Grummeltier commented 2 years ago

I have also an issue with IMAP import and latest linuxserver docker image. It will always import all new mails/documents within admin account, even the sender mails fits to an other user account and the "routing by user" is activates within config file, and within settings of admin and other user.

I have the same problem. Documents are forwarded to the superuser, even though the IMPORT_MAIL_BY_USER Settings is set True. The mail address of the user is the same als the send mail.

Mails are also complete ignored, when I send them from my iPhone.