glpi-project / glpi

GLPI is a Free Asset and IT Management Software package, Data center management, ITIL Service Desk, licenses tracking and software auditing.
https://glpi-project.org
GNU General Public License v3.0
4.24k stars 1.29k forks source link

Performance issue in ticket list related to `glpi_tickets`.`content` #6861

Closed mat44444 closed 4 years ago

mat44444 commented 4 years ago

Describe the bug

We use mail collector to received more than 95% of our ticket. Apparently, when we go in the list ticket page, the code automatically load in the sql request the glpi_tickets.content field. Probably for the preview feature because it was not set in the view.

Saddly, there a lot of html content due to the mailcollector and the requested take more then 5 second to load content. With debug we get this query who take more 99% of the load time.

SELECT DISTINCTglpi_tickets.idAS id, 'LecomptM' AS currentuser,glpi_tickets.idASITEM_Ticket_2,glpi_tickets.nameASITEM_Ticket_1,glpi_tickets.idASITEM_Ticket_1_id,glpi_tickets.idASITEM_Ticket_1_id,glpi_tickets.contentASITEM_Ticket_1_content,glpi_tickets.statusASITEM_Ticket_1_status,glpi_entities.completenameASITEM_Ticket_80,glpi_tickets.statusASITEM_Ticket_12,glpi_tickets.date_modASITEM_Ticket_19,glpi_tickets.dateASITEM_Ticket_15,glpi_itilcategories.completenameASITEM_Ticket_7, GROUP_CONCAT(DISTINCTglpi_users_f201be21cd638ee780d08cba4ceff0d4.idSEPARATOR '$$##$$') ASITEM_Ticket_5, GROUP_CONCAT(DISTINCT CONCAT(glpi_tickets_users_819efb92c8b927b345e489211ec8e43b.users_id, ' ',glpi_tickets_users_819efb92c8b927b345e489211ec8e43b.alternative_email) SEPARATOR '$$##$$') ASITEM_Ticket_5_2,glpi_tickets.priorityASITEM_Ticket_3 FROMglpi_tickets LEFT JOINglpi_entitiesON (glpi_tickets.entities_id=glpi_entities.id) LEFT JOINglpi_itilcategoriesON (glpi_tickets.itilcategories_id=glpi_itilcategories.id) LEFT JOINglpi_tickets_usersASglpi_tickets_users_819efb92c8b927b345e489211ec8e43bON (glpi_tickets.id=glpi_tickets_users_819efb92c8b927b345e489211ec8e43b.tickets_idANDglpi_tickets_users_819efb92c8b927b345e489211ec8e43b.type= 2 ) LEFT JOINglpi_usersASglpi_users_f201be21cd638ee780d08cba4ceff0d4ON (glpi_tickets_users_819efb92c8b927b345e489211ec8e43b.users_id=glpi_users_f201be21cd638ee780d08cba4ceff0d4.id) WHEREglpi_tickets.is_deleted= 0 AND (glpi_tickets.statusIN ('1','2','3','4') ) GROUP BYglpi_tickets.id ORDER BY ITEM_Ticket_19 DESC

Forcefully remove the glpi_tickets.content solve the performance issue.

So make an option to remove glpi_tickets.content call bye disable the preview feature. Or let the preview feature make is own call to sql when the mouse is over the link.

On our side the defaut parameter return around 500 ticket filtered by 25 tickets by page, but the request return the data of 500 tickets.

Page(s) URL

https://.../front/ticket.php

To reproduce

Steps to reproduce the behavior:

  1. Got a lot of ticket with heavy content in glpi_tickets.content
  2. Refresh the filter and let it load
  3. Look a the Debug for the loadtime of sql query

Expected behavior

Page should load more quickly

Screenshots

If applicable, add screenshots to help explain your problem.

Your GLPI setup (you can find it in Setup > General menu, System tab)

[code]   GLPI 9.4.4 ( => /opt/www/gri.dmz.urgences-sante.qc.ca/html) Installation mode: TARBALL

Operating system: Linux srv-pr-apachedmzv1.ipa.urgences-sante.qc.ca 3.10.0-1062.9.1.el7.x86_64 #1 SMP Fri Dec 6 15:49:49 UTC    2019 x86_64 PHP 7.1.33 fpm-fcgi (Core, PDO, Phar, Reflection, SPL, SimpleXML, Zend OPcache, apc, apcu, bz2, calendar, cgi-fcgi, ctype, curl,    date, dom, exif, fileinfo, filter, ftp, gd, gettext, hash, iconv, imap, intl, json, ldap, libxml, mbstring, mongodb, mysql,     mysqli, mysqlnd, openssl, pcre, pdo_mysql, pdo_pgsql, pdo_sqlite, pgsql, session, snmp, soap, sockets, sqlite3, standard,   tokenizer, wddx, xml, xmlreader, xmlrpc, xmlwriter, xsl, zip, zlib) Setup: max_execution_time="600" memory_limit="256M" post_max_size="512M" safe_mode="" session.save_handler="files"  upload_max_filesize="512M"  Software: Apache/2.4.41 (codeit) mod_fcgid/2.3.9 mod_perl/2.0.11 Perl/v5.16.3 ()    Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36 Server Software: MariaDB Server     Server Version: 10.2.30-MariaDB-log     Server SQL Mode:    Parameters: glpi_gri@gri.bd.interne.urgences-sante.qc.ca:3306/GRI   Host info: gri.bd.interne.urgences-sante.qc.ca via TCP/IP    mysqli extension is installed ctype extension is installed fileinfo extension is installed json extension is installed mbstring extension is installed iconv extension is installed zlib extension is installed curl extension is installed gd extension is installed simplexml extension is installed xml extension is installed ldap extension is installed imap extension is installed Zend OPcache extension is installed APCu extension is installed xmlrpc extension is installed exif extension is installed CAS extension is not present Database version seems correct (10.2.30) - Perfect! /opt/www/gri.dmz.urgences-sante.qc.ca/html/files/_log : OK /opt/www/gri.dmz.urgences-sante.qc.ca/html/config : OK /opt/www/gri.dmz.urgences-sante.qc.ca/html/files : OK /opt/www/gri.dmz.urgences-sante.qc.ca/html/files/_dumps : OK /opt/www/gri.dmz.urgences-sante.qc.ca/html/files/_sessions : OK /opt/www/gri.dmz.urgences-sante.qc.ca/html/files/_cron : OK /opt/www/gri.dmz.urgences-sante.qc.ca/html/files/_graphs : OK /opt/www/gri.dmz.urgences-sante.qc.ca/html/files/_lock : OK /opt/www/gri.dmz.urgences-sante.qc.ca/html/files/_plugins : OK /opt/www/gri.dmz.urgences-sante.qc.ca/html/files/_tmp : OK /opt/www/gri.dmz.urgences-sante.qc.ca/html/files/_cache : OK /opt/www/gri.dmz.urgences-sante.qc.ca/html/files/_rss : OK /opt/www/gri.dmz.urgences-sante.qc.ca/html/files/_uploads : OK /opt/www/gri.dmz.urgences-sante.qc.ca/html/files/_pictures : OK Web access to the files directory should not be allowed Check the .htaccess file and the web server configuration. SELinux mode is Disabled

htmLawed version 1.2.4 in (/opt/www/gri.dmz.urgences-sante.qc.ca/html/lib/htmlawed) phpmailer/phpmailer version 6.0.7 in (/opt/www/gri.dmz.urgences-sante.qc.ca/html/vendor/phpmailer/phpmailer/src) simplepie/simplepie version 1.5.2 in (/opt/www/gri.dmz.urgences-sante.qc.ca/html/vendor/simplepie/simplepie/library) tecnickcom/tcpdf version 6.2.26 in (/opt/www/gri.dmz.urgences-sante.qc.ca/html/vendor/tecnickcom/tcpdf) michelf/php-markdown in (/opt/www/gri.dmz.urgences-sante.qc.ca/html/vendor/michelf/php-markdown/Michelf) true/punycode in (/opt/www/gri.dmz.urgences-sante.qc.ca/html/vendor/true/punycode/src) iamcal/lib_autolink in (/opt/www/gri.dmz.urgences-sante.qc.ca/html/vendor/iamcal/lib_autolink) sabre/vobject in (/opt/www/gri.dmz.urgences-sante.qc.ca/html/vendor/sabre/vobject/lib) zendframework/zend-cache in (/opt/www/gri.dmz.urgences-sante.qc.ca/html/vendor/zendframework/zend-cache/src) zendframework/zend-i18n in (/opt/www/gri.dmz.urgences-sante.qc.ca/html/vendor/zendframework/zend-i18n/src) zendframework/zend-serializer in (/opt/www/gri.dmz.urgences-sante.qc.ca/html/vendor/zendframework/zend-serializer/src) monolog/monolog in (/opt/www/gri.dmz.urgences-sante.qc.ca/html/vendor/monolog/monolog/src/Monolog) sebastian/diff in (/opt/www/gri.dmz.urgences-sante.qc.ca/html/vendor/sebastian/diff/src) elvanto/litemoji in (/opt/www/gri.dmz.urgences-sante.qc.ca/html/vendor/elvanto/litemoji/src) symfony/console in (/opt/www/gri.dmz.urgences-sante.qc.ca/html/vendor/symfony/console) leafo/scssphp in (/opt/www/gri.dmz.urgences-sante.qc.ca/html/vendor/leafo/scssphp/src)

Server: 'ldap://shut.local', Port: '389', BaseDN: 'dc=urgences-sante,dc=local', Connection filter:      '(&(objectClass=user)(objectCategory=person)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))', RootDN:        'CN=secret,OU=System,OU=Comptes,DC=Urgences-sante,DC=local', Use TLS: none

Not active

Way of sending emails: SMTP (anonymous@ssmail02.urgences-sante.local)

Name: 'xx2@urgences-sante.qc.ca' Active: Yes    Server: '{ssmail02.urgences-sante.local/pop}INBOX' Login: 'urgences-sante\xx2' Password: Yes Name: 'Collecteur.GLPI@urgences-sante.qc.ca' Active: Yes   Server: '{ssmail02.urgences-sante.local/pop}INBOX' Login: 'xx3@urgences-sante.local' Password: Yes Name: 'xx1@urgences-sante.qc.ca' Active: Yes     Server: '{ssmail02.urgences-sante.local/pop}INBOX' Login: 'urgences-sante\xx1' Password: Yes

addressing           Name: Adressage IP                   Version: 2.8.0      State: Enabled    
additionalalerts     Name: Alertes supplémentaires        Version: 2.2.0      State: Not activated  
treeview             Name: Arborescence                   Version: 1.8.1      State: Enabled    
positions            Name: Cartographie                   Version: 4.6.0      State: Not activated  
servicecatalog       Name: Catalogue de service           Version: 1.6.1      State: Enabled    
shellcommands        Name: Commandes Shell                Version: 2.3.0      State: Not activated  
printercounters      Name: Compteurs Imprimantes          Version: 1.6.0      State: Enabled    
airwatch             Name: Connecteur Airwatch pour GLPi  Version: 1.4.0      State: Not activated  
mydashboard          Name: Dashboard                      Version: 1.7.6      State: Enabled    
consumables          Name: Demande de consommables        Version: 1.5.0      State: Not installed  
formcreator          Name: Form Creator                   Version: 2.8.6      State: Enabled    
fusioninventory      Name: FusionInventory                Version: 9.4+2.3    State: Enabled    
genericobject        Name: Gestion d'objets               Version: 2.7.0      State: Not activated  
datainjection        Name: Injection de fichiers          Version: 2.7.0      State: Enabled    
tasklists            Name: Liste des tâches               Version: 1.5.0      State: Not activated  
mailanalyzer         Name: Mail Analyzer                  Version: 1.6.1      State: Enabled    
mreporting           Name: Plus de rapports               Version: 1.6.1      State: Enabled    
protocolsmanager     Name: Protocols manager              Version: 1.0        State: Enabled    
resources            Name: Ressources humaines            Version: 2.6.1      State: Not installed  
dashboard            Name: Tableau de bord                Version: 0.9.8      State: Enabled    
typology             Name: Typologies                     Version: 2.6.0      State: Enabled
[/code]

Additional context

Add any other context about the problem here.

trasher commented 4 years ago

Informations requested in ticket template has not been provided.

mat44444 commented 4 years ago

GLPI setup added

cedric-anne commented 4 years ago

Hi,

Technically, it could be improved by doing 2 queries instead of 1. The first one to count items matching search criteria (only a SELECT COUNT with all criteria to apply), and a second one to fetch only displayed elements. But it is actually really hard to implement as code of search engine is nor well documented, nor well automatically tested.

But, unless content of your tickets are really huge, you should not face such performances issues with only 500 tickets in your DB. For instance, I made a test on a production server, and it tooks me only 1.72 s to display a list of 50 tickets, sorted by their name, from a database containing 18500 tickets. So you should try to tweak your database server.

As this is not a bug, and as performances enhancement on this point will not be adressed soon, I close the issue.

Feel free to ask for community support on forums.

Regards