[9.2.1] mailgate and queuednotification tasks remain running

ppenin commented 6 years ago

In GLPI 9.2.1, when a mail with a picture embedded in the body, the mailgate and queuednotification tasks remain running till we blank them with the cross. Then the mail is collected but 2 tickets are created. This issue never happened in older versions, 9.1.6 included. I started with 9.2 Nothing in the logs (cron.log, php-errors.log, mail.log). the issue is very boring because we have to blank the tasks many times a day.

GLPI 9.2.1 (/glpi => /usr/share/glpi) Installation mode: TARBALL

Operating system: Linux s001linux02 4.4.0-112-generic #135-Ubuntu SMP Fri Jan 19 11:48:36 UTC 2018 x86_64 PHP 7.0.26-1+ubuntu16.04.1+deb.sury.org+1 apache2handler (Core, PDO, Phar, Reflection, SPL, SimpleXML, Zend OPcache, apache2handler, calendar, ctype, curl, date, dom, exif, fileinfo, filter, ftp, gd, gettext, hash, iconv, imap, json, ldap, libxml, mbstring, mysqli, mysqlnd, openssl, pcre, pdo_mysql, posix, readline, session, shmop, sockets, standard, sysvmsg, sysvsem, sysvshm, tokenizer, wddx, xml, xmlreader, xmlwriter, xsl, zlib) Setup: max_execution_time="30" memory_limit="256M" post_max_size="100M" safe_mode="" session.save_handler="files" upload_max_filesize="100M" Software: Apache/2.4.18 (Ubuntu) (Apache/2.4.18 (Ubuntu) Server at 10.100.10.18 Port 80) Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0 Server Software: (Ubuntu) Server Version: 5.7.21-0ubuntu0.16.04.1 Server SQL Mode: Parameters: root@localhost/glpi Host info: Localhost via UNIX socket mysqli extension is installed ctype extension is installed fileinfo extension is installed json extension is installed mbstring extension is installed zlib extension is installed curl extension is installed gd extension is installed simplexml extension is installed xml extension is installed ldap extension is installed imap extension is installed Zend OPcache extension is installed APCu extension is not present xmlrpc extension is not present /usr/share/glpi/config : OK /usr/share/glpi/files : OK /usr/share/glpi/files/_dumps : OK /usr/share/glpi/files/_sessions : OK /usr/share/glpi/files/_cron : OK /usr/share/glpi/files/_graphs : OK /usr/share/glpi/files/_lock : OK /usr/share/glpi/files/_plugins : OK /usr/share/glpi/files/_tmp : OK /usr/share/glpi/files/_cache : OK /usr/share/glpi/files/_rss : OK /usr/share/glpi/files/_uploads : OK /usr/share/glpi/files/_pictures : OK /usr/share/glpi/files/_log : OK Web access to files directory is protected : OK

htmLawed version 1.2.4 in (/usr/share/glpi/lib/htmlawed) phpCas version 1.3.5 in (/usr/share/glpi/vendor/jasig/phpcas/source) PHPMailer version 5.2.26 in (/usr/share/glpi/vendor/phpmailer/phpmailer) SimplePie version 1.5.1 in (/usr/share/glpi/vendor/simplepie/simplepie/library) TCPDF version 6.2.13 in (/usr/share/glpi/vendor/tecnickcom/tcpdf) michelf/php-markdown in (/usr/share/glpi/vendor/michelf/php-markdown/Michelf) true/punycode in (/usr/share/glpi/vendor/true/punycode/src) iacaml/autolink in (/usr/share/glpi/vendor/iamcal/lib_autolink) sabre/vobject in (/usr/share/glpi/vendor/sabre/vobject/lib)

Server: '10.100.10.1', Port: '389', BaseDN: 'DC=mydomain,DC=priv', Connection filter: none, RootDN: 'CN=Service Alerte,OU=Comptes de Services,OU=001 (FSS),DC=mydomain,DC=priv', Use TLS: none

Not active

Way of sending emails: SMTP (hotline-glpi@10.100.10.66)

Name: 'hotline@mydomain.fr' Active: Yes Server: '{10.100.10.66/imap/novalidate-cert/tls}INBOX' Login: 'hotline-GLPI' Password: Yes

fields Name: Champs supplémentaires Version: 1.7.0 State: Enabled escalade Name: Escalades Version: 2.2.1 State: Enabled simcard Name: Gestion de cartes SIM Version: 1.4.2 State: Enabled pdf Name: Impression pdf Version: 1.3.0 State: Enabled datainjection Name: Injection de fichiers Version: 2.5.0 State: Enabled ocsinventoryng Name: OCS Inventory NG Version: 1.4.2 State: Enabled mreporting Name: Plus de rapports Version: 1.4.0 State: Enabled dashboard Name: Tableau de bord Version: 0.9.0 State: Enabled

Host: localhostConnection: OkUse the OCSNG software dictionary: No

ppenin commented 6 years ago

the only backtrace found in php-errors.log is 018-02-01 11:56:02 [@s001linux02] *** PHP Notice(8): Undefined property: stdClass::$bytes Backtrace : inc/mailcollector.class.php:1495
inc/mailcollector.class.php:1413 MailCollector->getRecursiveAttached() inc/mailcollector.class.php:1551 MailCollector->getRecursiveAttached() inc/mailcollector.class.php:704 MailCollector->getAttached() inc/mailcollector.class.php:539 MailCollector->buildTicket() inc/mailcollector.class.php:1659 MailCollector->collect() inc/crontask.class.php:832 MailCollector::cronMailgate() front/cron.php:87 CronTask::launch()

trasher commented 6 years ago

You should try with 9.2.2

ppenin commented 6 years ago

Hello, It has been tested with 9.2.2 for a couple of months. Same issue.

ppenin commented 6 years ago

I'm waiting for the release 9.2.3 to make new tests. My mind is that the issue depends on Exchange Server 2013 POP3 configuration

ppenin commented 6 years ago

[9.3.0] same issue with mailgate and queuednotification still remaining running when a mail with a picture embedded is collected.

trasher commented 6 years ago

I'm not able to reproduce the issue. It may be related to the image itself (too big? broken?) or to the way it's been included in the message or to the connection with the collector (it will be very hard to reproduce/fix in that case). Could you please send a message that fails to glpi-debug AT teclib. DOT com, just to be sure it works on my side?

rjmontenegro commented 6 years ago

I'll check it out better but in recent weeks the queue of notifications got huge notifications to send and the automatic queuednotification task was always running. will not be related to these problems that I also had: https://github.com/glpi-project/glpi/issues/1929 https://github.com/glpi-project/glpi/issues/3653 ?

trasher commented 6 years ago

@ppenin thanks for the mail, but "msg" files are unusable to me; I need to get the message to import in the inbox (the one you sent has been imported sucessfully). @rjmontenegro 1929 and 3653 are the same; and this has been fixed (was not blocking collector anyways).

ppenin commented 6 years ago

I dunno how to send you mails without transforming them. Maybe you'd like to analyze the headers ? The scenario is allways the same

The mail is collected (POP3) and a new ticket is created in GLPI
The mail remains in Inbox (it looks like an unread mail)
Mailgate Task stucks
Queuednotification task stucks We have to click on the cross and run manually the tasks Plugin Mail analyzer avoids to get a second duplicate ticket. I tested without the plugin and same issue. We never has these issues in 9.16 and earlier. Something has changed since but i dunno what.

trasher commented 6 years ago

I need the faultly mail to arrive in the collector inbox; just as you revceive iit on your side. Most of the time, there is a content issue due to the original software. I'm not able to reproduce on my side (tested with imap and pop , with th mail you sent and others: all goes well, images are present and crontask ends fine.

Something has changed since but i dunno what.

Many, many things...

trasher commented 6 years ago

Just tested with latest received mails, still cannot reproduce. You'll have to find on your side why this one is not imported. Try with debug mode enabled (if you run import from the UI); double check the logs (GLPI and system PHP). Maybe mail server logs can help as well.

nectop commented 6 years ago

Hello everyone. Same issue here. Fresh install in GLPI 9.3. I found, that when GLPI parses email from hub it save image to _files\tmp and then do not insert it to the ticket. Also i temporary gave 777 to _files folder to get tid of mailgate stuck issue.

nectop commented 6 years ago

some updates. We have 9.1.1 that is working. I tried to upade it to any of release from 9.1.6 to 9.2.3/4 or 9.3 Only 9.1.6 doesn't have pictures in mail issue.

imiddleton1 commented 6 years ago

I am experiencing the same issue with version 9.3. I recent;y upgraded from a very old version (0.84) that did not have this problem with similar content in e-mail (I had been using 0.84 since 2014 with no mailgate issues)

Our company mandated e-mail signatures have small facebook. twitter and instagram logos in everyone's signature as well as the company logo. All logo sizes are small. The social media logos are all 0.3" and the company logo is 0.5" x 1.5" (actual size).

cedric-anne commented 6 years ago

Hello everyone,

On which server are you running GLPI (Windows, Linux, ...) ? How do you run the mailgate task ? In GLPI mode, or CLI mode ? Which rights does have user launching cron on _files and _files/_tmp directories ? Do you have something in your logs ?

Regards

ppenin commented 5 years ago

Sorry, i'm late... Technical informations bellow Mailtask and queuednotifiation tasks are in CLI mode /_files and /_files/tmp : 775 nothing in the PHP and MySql logs. We tested with a new collector and a Gmail address and it works. With a POP3 address on our Exchange Server, the tasks stuck when an mail contains 1 or more pictures embedded. Nothing in the eventvwr. It never happened in 9.1.6 and issue started with 9.2

================ GLPI 9.3.2 (/glpi => /usr/share/glpi) Installation mode: TARBALL

Server

Operating system: Linux s001linux02 4.4.0-134-generic #160-Ubuntu SMP Wed Aug 15 14:58:00 UTC 2018 x86_64 PHP 7.0.32-3+ubuntu16.04.1+deb.sury.org+1 apache2handler (Core, PDO, Phar, Reflection, SPL, SimpleXML, Zend OPcache, apache2handler, apc, apcu, calendar, ctype, curl, date, dom, exif, fileinfo, filter, ftp, gd, gettext, hash, iconv, imap, json, ldap, libxml, mbstring, mysqli, mysqlnd, openssl, pcre, pdo_mysql, posix, readline, session, shmop, sockets, standard, sysvmsg, sysvsem, sysvshm, tokenizer, wddx, xml, xmlreader, xmlrpc, xmlwriter, xsl, zlib) Setup: max_execution_time="30" memory_limit="1024M" post_max_size="1000M" safe_mode="" session.save_handler="files" upload_max_filesize="1000M" Software: Apache/2.4.18 (Ubuntu) (Apache/2.4.18 (Ubuntu) Server at x.x.x.x8 Port 80) Mozilla/5.0 (Windows NT 6.1; WOW64; rv:60.0) Gecko/20100101 Firefox/60.0 Server Software: (Ubuntu) Server Version: 5.7.24-0ubuntu0.16.04.1 Server SQL Mode: Parameters: root@localhost/glpi Host info: Localhost via UNIX socket

mysqli extension is installed ctype extension is installed fileinfo extension is installed json extension is installed mbstring extension is installed zlib extension is installed curl extension is installed gd extension is installed simplexml extension is installed xml extension is installed ldap extension is installed imap extension is installed Zend OPcache extension is installed APCu extension is installed xmlrpc extension is installed CAS extension is not present Database version seems correct (5.7.24) - Perfect!Database version seems correct (5.7.24) - Perfect! OK/usr/share/glpi/files/_log : OK OK/usr/share/glpi/config : OK OK/usr/share/glpi/files : OK OK/usr/share/glpi/files/_dumps : OK OK/usr/share/glpi/files/_sessions : OK OK/usr/share/glpi/files/_cron : OK OK/usr/share/glpi/files/_graphs : OK OK/usr/share/glpi/files/_lock : OK OK/usr/share/glpi/files/_plugins : OK OK/usr/share/glpi/files/_tmp : OK OK/usr/share/glpi/files/_cache : OK OK/usr/share/glpi/files/_rss : OK OK/usr/share/glpi/files/_uploads : OK OK/usr/share/glpi/files/_pictures : OK Web access to files directory is protectedWeb access to files directory is protected : OK

Libraries

htmLawed version 1.2.4 in (/usr/share/glpi/lib/htmlawed) SimplePie version 1.5.2 in (/usr/share/glpi/vendor/simplepie/simplepie/library) TCPDF version 6.2.25 in (/usr/share/glpi/vendor/tecnickcom/tcpdf) michelf/php-markdown in (/usr/share/glpi/vendor/michelf/php-markdown/Michelf) true/punycode in (/usr/share/glpi/vendor/true/punycode/src) iacaml/autolink in (/usr/share/glpi/vendor/iamcal/lib_autolink) sabre/vobject in (/usr/share/glpi/vendor/sabre/vobject/lib)

LDAP directories

Server: 'x.x.x.x', Port: '389', BaseDN: 'DC=mydomain,DC=priv', Connection filter: none, RootDN: 'CN=zzz,OU=Comptes de Services,OU=001,DC=mydomain,DC=priv', Use TLS: none

SQL replicas

Not active

Notifications

Way of sending emails: SMTP (hotline-glpi@x.x.x.x)

Mails receivers

Name: 'hotline@mydomain.fr' Active: Yes Server: '{x.x.x.x/pop/novalidate-cert/notls/debug}INBOX' Login: 'hotline-GLPI' Password: Yes Name: 'rgpd@mydomain.fr' Active: No Server: '{x.x.x.x/pop/novalidate-cert/notls}INBOX' Login: 'contact-rgpd' Password: Yes Name: 'test.xxx@gmail.com' Active: No Server: '{imap.gmail.com:993/imap/ssl/debug}INBOX' Login: 'test.mydomain@gmail.com' Password: Yes

Plugins list

barcode              Name: Barcode                        Version: 2.2.1      State: Enabled
fields               Name: Champs supplémentaires         Version: 1.8.2      State: Enabled
timelineticket       Name: Chronologie des tickets        Version: 9.3+1.0    State: Enabled
escalade             Name: Escalades                      Version: 2.3.3      State: Enabled
fusioninventory      Name: FusionInventory                Version: 9.3+1.1    State: Enabled
pdf                  Name: Impression pdf                 Version: 1.4.0      State: Enabled
datainjection        Name: Injection de fichiers          Version: 2.6.4      State: Enabled
mailanalyzer         Name: Mail Analyzer                  Version: 1.3.8      State: Enabled
ocsinventoryng       Name: OCS Inventory NG               Version: 1.5.4      State: Enabled
mreporting           Name: Plus de rapports               Version: 1.5.3      State: Enabled
reports              Name: Rapports                       Version: 1.12.0     State: Enabled
dporegister          Name: Registre DPO                   Version: 1.3.1      State: Enabled
dashboard            Name: Tableau de bord                Version: 0.9.4      State: Enabled

OCS Inventory NG Host: localhost Connection: Ok Use the OCSNG software dictionary: No

cedric-anne commented 5 years ago

Hi,

We have no access to any Exchange server, so it is really complicated to debug, especially without any error trace in the logs.

Regards

ppenin commented 5 years ago

It’s very annoying because we spend our time to release mailgate tasks for many months. I have no idea how to solve this issue. Would it be possible for you to have a remote connection to watch this issue ?

Le 6 nov. 2018 à 17:18, Cédric Anne notifications@github.com<mailto:notifications@github.com> a écrit :

Hi,

We have no access to any Exchange server, so it is really complicated to debug, especially without logs.

Regards

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/glpi-project/glpi/issues/3494#issuecomment-436312563, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AVdRNgc9i0Sv89xS8KwJOoVP_dcuxBdPks5usbZegaJpZM4R1SSF.

cedric-anne commented 5 years ago

Hi,

It could be possible in the context of the professional support. You should take a look at https://services.glpi-network.com/ .

Regards

cedric-anne commented 5 years ago

Hi @ppenin ,

We made a change in 9.4/bugfixes branch that may help to understand why your mailgate task stay stucked. Indeed, no trace was displayed if a fatal error occurs when launching the front/cron.php script from command line. This has been changed by #5578 .

We also made some fixes that can affect the mailcollector behaviour since 9.3.2.

Could you make a test by launching manually the cron from CLI using the 9.4 nightly build to see if the problem still occurs ?

Regards

ppenin commented 5 years ago

Mailgate and queuednotications tasks aren’t stuck anymore. We don’t have any explanation about it. But everything works fine now. The issue can be closed.

cedric-anne commented 5 years ago

Glad your problem is solved.

We fixed many problems on the collector in 9.3.x and 9.4.x, so I close this issue, assuming that different problems described here are fixed.

Regards

glpi-project / glpi

[9.2.1] mailgate and queuednotification tasks remain running #3494