laszlokertesz
commented
6 months ago I found the log entries. It does seem to point to the glpi inventory plugin?
/var/www/html/glpi/files/_log/sql-errors.log: SQL: DELETE `glpi_logs` FROM `glpi_logs` WHERE `itemtype` = 'Agent' AND `items_id` = '1051'
/var/www/html/glpi/files/_log/sql-errors.log- Error: Lock wait timeout exceeded; try restarting transaction
/var/www/html/glpi/files/_log/sql-errors.log- Backtrace :
/var/www/html/glpi/files/_log/sql-errors.log- src/DBmysql.php:1579
/var/www/html/glpi/files/_log/sql-errors.log- src/CommonDBTM.php:841 DBmysql->delete()
/var/www/html/glpi/files/_log/sql-errors.log- src/CommonDBTM.php:787 CommonDBTM->cleanHistory()
/var/www/html/glpi/files/_log/sql-errors.log- src/CommonDBTM.php:2068 CommonDBTM->deleteFromDB()
/var/www/html/glpi/files/_log/sql-errors.log- src/CommonDBTM.php:4627 CommonDBTM->delete()
/var/www/html/glpi/files/_log/sql-errors.log- src/Inventory/Asset/MainAsset.php:786 CommonDBTM->deleteByCriteria()
/var/www/html/glpi/files/_log/sql-errors.log- src/RuleImportAsset.php:969 Glpi\Inventory\Asset\MainAsset->rulepassed()
/var/www/html/glpi/files/_log/sql-errors.log- src/Rule.php:1525 RuleImportAsset->executeActions()
/var/www/html/glpi/files/_log/sql-errors.log- src/RuleCollection.php:1640 Rule->process()
/var/www/html/glpi/files/_log/sql-errors.log- src/Inventory/Asset/MainAsset.php:573 RuleCollection->processAllRules()
/var/www/html/glpi/files/_log/sql-errors.log- src/Inventory/Inventory.php:709 Glpi\Inventory\Asset\MainAsset->handle()
/var/www/html/glpi/files/_log/sql-errors.log- src/Inventory/Inventory.php:340 Glpi\Inventory\Inventory->handleItem()
/var/www/html/glpi/files/_log/sql-errors.log- src/Inventory/Request.php:360 Glpi\Inventory\Inventory->doInventory()
/var/www/html/glpi/files/_log/sql-errors.log- src/Inventory/Request.php:90 Glpi\Inventory\Request->inventory()
/var/www/html/glpi/files/_log/sql-errors.log- src/Agent/Communication/AbstractRequest.php:359 Glpi\Inventory\Request->handleAction()
/var/www/html/glpi/files/_log/sql-errors.log- src/Agent/Communication/AbstractRequest.php:271 Glpi\Agent\Communication\AbstractRequest->handleJSONRequest()
/var/www/html/glpi/files/_log/sql-errors.log- front/inventory.php:92 Glpi\Agent\Communication\AbstractRequest->handleRequest()
/var/www/html/glpi/files/_log/sql-errors.log- plugins/glpiinventory/front/communication.php:72 include_once()
/var/www/html/glpi/files/_log/sql-errors.log- plugins/glpiinventory/index.php:45 include_once()
/var/www/html/glpi/files/_log/sql-errors.log- {"user":"inventory@inventory.3pillarglobal.com","mem_usage":"87.900\", 6.13Mio)"}
stonebuzz
cconard96
github-actions[bot]
Code of Conduct
Is there an existing issue for this?
Version
10.0.12
Bug description
After reboot of the server and start of apache and mariadb (this happened both with 10.0.9 and 10.0.12) the database becomes flooded with queries like
I counted 350 at a certain point. This is a big issue since our logs table is really big (around 800 000 000 records) and these queries bog the machine down until the service basically becomes unresponsive. These are related to incoming inventories (if i disable it, after a while the server becomes serviceable), i presume every agent that reports in gets a query like this. But these take very long time, sometimes they get killed because of timeouts and result in rollbacks.
I don't know why these are triggered, the agents that appear on the list are both new and old. Can someone give some pointers where can i disable this, i can do some background cleanup if needed but tying these queries to inventory requests that result in the web server being overwhelmed is not really working out.
Relevant log output
No response
Page URL
No response
Steps To reproduce
I just restarted the server (had some 70 days uptime) and this behavior started.
Your GLPI setup information
Information about system installation and configuration
Server
Operating system: Linux inventory.3pillarglobal.com 6.1.0-12-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.52-1 (2023-09-07) x86_64 PHP 8.2.7 apache2handler (Core, FFI, PDO, Phar, Reflection, SPL, SimpleXML, Zend OPcache, apache2handler, apcu, bz2, calendar, ctype, curl, date, dom, exif, fileinfo, filter, ftp, gd, gettext, hash, iconv, imap, intl, json, ldap, libxml, mbstring, mysqli, mysqlnd, openssl, pcre, pdo_mysql, posix, random, readline, session, shmop, sockets, sodium, ssh2, standard, sysvmsg, sysvsem, sysvshm, tokenizer, xml, xmlreader, xmlrpc, xmlwriter, xsl, zip, zlib) Setup: max_execution_time="30" memory_limit="256M" post_max_size="8M" safe_mode="" session.save_handler="files" upload_max_filesize="2M" Software: Apache () Mozilla/5.0 (X11; Linux x86_64; rv:122.0) Gecko/20100101 Firefox/122.0 Server Software: Debian 12 Server Version: 10.11.3-MariaDB-1 Server SQL Mode: STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION Parameters: root@localhost/glpi Host info: Localhost via UNIX socket PHP version (8.2.7) is supported. Sessions configuration is OK. Allocated memory is sufficient. mysqli extension is installed. Following extensions are installed: dom, fileinfo, filter, libxml, json, simplexml, xmlreader, xmlwriter. curl extension is installed. gd extension is installed. intl extension is installed. zlib extension is installed. The constant SODIUM_CRYPTO_AEAD_XCHACHA20POLY1305_IETF_NPUBBYTES is present. Database engine version (10.11.3) is supported. No files from previous GLPI version detected. The log file has been created successfully. Write access to /var/www/html/glpi/files/_cache has been validated. Write access to /var/www/html/glpi/config has been validated. Write access to /var/www/html/glpi/files/_cron has been validated. Write access to /var/www/html/glpi/files has been validated. Write access to /var/www/html/glpi/files/_dumps has been validated. Write access to /var/www/html/glpi/files/_graphs has been validated. Write access to /var/www/html/glpi/files/_lock has been validated. Write access to /var/www/html/glpi/files/_pictures has been validated. Write access to /var/www/html/glpi/files/_plugins has been validated. Write access to /var/www/html/glpi/files/_rss has been validated. Write access to /var/www/html/glpi/files/_sessions has been validated. Write access to /var/www/html/glpi/files/_tmp has been validated. Write access to /var/www/html/glpi/files/_uploads has been validated. Web server root directory configuration seems safe. PHP directive "session.cookie_secure" should be set to "on" when GLPI can be accessed on HTTPS protocol. PHP directive "session.cookie_httponly" should be set to "on" to prevent client-side script to access cookie values. OS and PHP are relying on 64 bits integers. exif extension is installed. ldap extension is installed. openssl extension is installed. Following extensions are installed: bz2, Phar, zip. Zend OPcache extension is installed. Following extensions are installed: ctype, iconv, mbstring, sodium. Write access to /var/www/html/glpi/marketplace has been validated. Timezones seems loaded in database.GLPI constants
Libraries
LDAP directories
Server: 'ldaps://1.1.1.1', Port: '636', BaseDN: 'DC=domain,DC=corp', Connection filter: '(&(objectClass=user)(objectCategory=person)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))', RootDN: 'CN=Glpi Auth,OU=AD,OU=ServiceAccounts,OU=AllUsers,DC=domain,DC=corp', Use TLS: none Server: 'localhost', Port: '1636', BaseDN: 'DC=domain,DC=com', Connection filter: none, RootDN: 'OptimalGre', Use TLS: noneSQL replicas
Notifications
Plugins list
glpiinventory Name: GLPI Inventory Version: 1.2.3 State: Enabled Install Method: Manual phpsaml Name: PHP SAML Version: 1.2.1 State: Enabled Install Method: ManualAnything else?
We have around 3000 endpoints reporting. The behavior started after upgrading to 10.0.12, but even after a rollback to a backup with 10.0.9 and reboot it did the same so this is not tied to version 10.0.12, but to something else, some cleanup routine i guess but i haven't found it's source.