search

jcreigno / nodejs-mail-notifier

nodejs library to listen incoming mail
MIT License
168 stars 77 forks source link

duplicate email notification #38

Closed sdejean28 closed 7 years ago

sdejean28 commented 7 years ago

i'm using the module to check an exchange mail box, but on 1000/1200 messages received per day, i've got more than 100/150 duplicates notifications. anyone the same issue ? thanks

inselberg commented 7 years ago

As long as they are marked as unread they throw notifications. Collect and check the messageIds.

sdejean28 commented 7 years ago

messageIds identical ... very weird ! may be when there are a lot of email it received a 'mail' event for each and can make 2 scan() simultaneously ... maybe we should allow only one scan at a time ...

sdejean28 commented 7 years ago

here's log that show that i scan at the same times

<= 'IDLE OK IDLE completed.' => 'A206 UID SEARCH UNSEEN' <= ' SEARCH 358256 358257' <= 'A206 OK SEARCH completed.' => 'A207 UID SEARCH UNSEEN' <= ' SEARCH 358256 358257' <= 'A207 OK SEARCH completed.' => 'A208 UID FETCH 358256,358257 (UID FLAGS INTERNALDATE BODY[])' <= ' 2209 FETCH (UID 358256 FLAGS (\Seen \Recent) INTERNALDATE "12-Sep-2017 09:32:40 +0200" BODY[] {11246}' <= ' 2209 FETCH (UID 358256 FLAGS (\Seen \Recent) INTERNALDATE "12-Sep-2017 09:32:40 +0200" FLAGS (\Seen \Recent))' <= ' 2210 FETCH (UID 358257 FLAGS (\Seen \Recent) INTERNALDATE "12-Sep-2017 09:32:55 +0200" BODY[] {21312}' <= ' 2210 FETCH (UID 358257 FLAGS (\Seen \Recent) INTERNALDATE "12-Sep-2017 09:32:55 +0200" FLAGS (\Seen \Recent))' <= 'A208 OK FETCH completed.' => 'A209 UID FETCH 358256,358257 (UID FLAGS INTERNALDATE BODY[])' 12 Sep 07:32:57 - Done fetching all messages! <= ' 2209 FETCH (UID 358256 FLAGS (\Seen \Recent) INTERNALDATE "12-Sep-2017 09:32:40 +0200" BODY[] {11246}' <= ' 2209 FETCH (UID 358256 FLAGS (\Seen \Recent) INTERNALDATE "12-Sep-2017 09:32:40 +0200" FLAGS (\Seen \Recent))' <= ' 2210 FETCH (UID 358257 FLAGS (\Seen \Recent) INTERNALDATE "12-Sep-2017 09:32:55 +0200" BODY[] {21312}' <= ' 2210 FETCH (UID 358257 FLAGS (\Seen \Recent) INTERNALDATE "12-Sep-2017 09:32:55 +0200" FLAGS (\Seen \Recent))' <= 'A209 OK FETCH completed.' => 'IDLE IDLE' 12 Sep 07:32:57 - Done fetching all messages!

jcreigno commented 7 years ago

Have you lost connection the underlying imap connection or restarted the listener ?

sdejean28 commented 7 years ago

no connection lost, everything is normal. I've implemented yesterday a queue manager (with async.js/queue) and now it works well. i really think that we receive an imap notification for each message, and so we perform a simultaneous search and so for each search we've got the message ...

jcreigno commented 7 years ago

@sdejean28 I've pushed a branch avoid-multiple-scan to address this specific issue. I think the issue is related to the scan triggered by the mail event. I suspect that the mail event is fired on the imap connection before the current scan has ended.

Can you try if this branch is fixing your issue ? If it does I'll merge it into the master.

Thanks

jcreigno commented 7 years ago

I'm a little reluctant to add async (like in your PR #40) that's why I've created a branch to play around. But after digging into this issue it seems that I need to use some kind of queue to prevent concurrent scans.