GLPI 10.0.1 - Glpi inventory spam logs, creating thousands of entries, same device move all the time between computers and changes manufacturer

[SteadEXE]

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

Is there an existing issue for this?

• [X] I have searched the existing issues

Version

10.0.1

Bug description

I discovered that everytime an agent inventories a computer a lot of USB/Network Interface devices get added and immediately deleted. It creates thousands lines of logs everyday. At the moment I added a purge task to prevent database from growing.

It appears to occurs for docks/usb devices. Maybe there is a way to identify them in a more reliable way.

My computer (another example):

Relevant log output

No response

Page URL

No response

Steps To reproduce

No response

Your GLPI setup information

Informations sur le système, l'installation et la configuration

GLPI 10.0.1 ( => E:\GLPI)
Installation mode: TARBALL
Current language:fr_FR

Server

 
Operating system: Windows NT SRVGLPI10 10.0 build 17763 (Windows Server 2019) AMD64
PHP 8.1.6 cgi-fcgi (Core, PDO, Phar, Reflection, SPL, SimpleXML, Zend OPcache, bcmath, bz2, calendar, cgi-fcgi, ctype, curl,
    date, dom, exif, fileinfo, filter, gd, hash, iconv, intl, json, ldap, libxml, mbstring, mysqli, mysqlnd, openssl, pcre,
    readline, session, sodium, standard, tokenizer, xml, xmlreader, xmlwriter, zip, zlib)
Setup: max_execution_time="30" memory_limit="256M" post_max_size="2048M" safe_mode="" session.save_handler="files"
    upload_max_filesize="2048M" 
Software: Microsoft-IIS/10.0
    Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.61 Safari/537.36
Server Software: mariadb.org binary distribution
    Server Version: 10.8.3-MariaDB
    Server SQL Mode: STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
    Parameters: glpi@localhost/glpi
    Host info: localhost via TCP/IP

PHP version (8.1.6) is supported.
Sessions configuration is OK.
Allocated memory is sufficient.
mysqli extension is installed.
Following extensions are installed: dom, fileinfo, json, simplexml.
curl extension is installed.
gd extension is installed.
intl extension is installed.
libxml extension is installed.
zlib extension is installed.
The constant SODIUM_CRYPTO_AEAD_XCHACHA20POLY1305_IETF_NPUBBYTES is present.
Database engine version (10.8.3) is supported.
The log file has been created successfully.
Write access to E:\GLPI/files/_cache has been validated.
Write access to E:\GLPI/config has been validated.
Write access to E:\GLPI/files/_cron has been validated.
Write access to E:\GLPI/files has been validated.
Write access to E:\GLPI/files/_dumps has been validated.
Write access to E:\GLPI/files/_graphs has been validated.
Write access to E:\GLPI/files/_lock has been validated.
Write access to E:\GLPI/files/_pictures has been validated.
Write access to E:\GLPI/files/_plugins has been validated.
Write access to E:\GLPI/files/_rss has been validated.
Write access to E:\GLPI/files/_sessions has been validated.
Write access to E:\GLPI/files/_tmp has been validated.
Write access to E:\GLPI/files/_uploads has been validated.
Web access to files directory is protected
exif extension is installed.
ldap extension is installed.
openssl extension is installed.
zip extension is installed.
bz2 extension is installed.
Zend OPcache extension is installed.
Following extensions are installed: ctype, iconv, mbstring, sodium.
Write access to E:\GLPI/marketplace has been validated.
Timezones seems loaded in database.

GLPI constants

 
GLPI_ROOT: E:\GLPI
GLPI_CONFIG_DIR: E:\GLPI/config
GLPI_VAR_DIR: E:\GLPI/files
GLPI_MARKETPLACE_DIR: E:\GLPI/marketplace
GLPI_USE_CSRF_CHECK: 1
GLPI_CSRF_EXPIRES: 7200
GLPI_CSRF_MAX_TOKENS: 100
GLPI_USE_IDOR_CHECK: 1
GLPI_IDOR_EXPIRES: 7200
GLPI_ALLOW_IFRAME_IN_RICH_TEXT: 
GLPI_TELEMETRY_URI: https://telemetry.glpi-project.org
GLPI_INSTALL_MODE: TARBALL
GLPI_NETWORK_MAIL: glpi@teclib.com
GLPI_NETWORK_SERVICES: https://services.glpi-network.com
GLPI_MARKETPLACE_ALLOW_OVERRIDE: 1
GLPI_MARKETPLACE_MANUAL_DOWNLOADS: 1
GLPI_USER_AGENT_EXTRA_COMMENTS: 
GLPI_DISABLE_ONLY_FULL_GROUP_BY_SQL_MODE: 1
GLPI_AJAX_DASHBOARD: 1
GLPI_CALDAV_IMPORT_STATE: 0
GLPI_DEMO_MODE: 0
GLPI_CENTRAL_WARNINGS: 1
GLPI_DOC_DIR: E:\GLPI/files
GLPI_CACHE_DIR: E:\GLPI/files/_cache
GLPI_CRON_DIR: E:\GLPI/files/_cron
GLPI_DUMP_DIR: E:\GLPI/files/_dumps
GLPI_GRAPH_DIR: E:\GLPI/files/_graphs
GLPI_LOCAL_I18N_DIR: E:\GLPI/files/_locales
GLPI_LOCK_DIR: E:\GLPI/files/_lock
GLPI_LOG_DIR: E:\GLPI/files/_log
GLPI_PICTURE_DIR: E:\GLPI/files/_pictures
GLPI_PLUGIN_DOC_DIR: E:\GLPI/files/_plugins
GLPI_RSS_DIR: E:\GLPI/files/_rss
GLPI_SESSION_DIR: E:\GLPI/files/_sessions
GLPI_TMP_DIR: E:\GLPI/files/_tmp
GLPI_UPLOAD_DIR: E:\GLPI/files/_uploads
GLPI_INVENTORY_DIR: E:\GLPI/files/_inventories
GLPI_NETWORK_REGISTRATION_API_URL: https://services.glpi-network.com/api/registration/
GLPI_MARKETPLACE_PLUGINS_API_URI: https://services.glpi-network.com/api/glpi-plugins/
GLPI_I18N_DIR: E:\GLPI/locales
GLPI_VERSION: 10.0.1
GLPI_SCHEMA_VERSION: 10.0.1@a130db99c7d9b131c2e2ea59fe0d6260fe93d831
GLPI_MARKETPLACE_PRERELEASES: 
GLPI_MIN_PHP: 7.4.0
GLPI_MAX_PHP: 8.2.0
GLPI_YEAR: 2022

Libraries

 
htmlawed/htmlawed version 1.2.6 in (E:\GLPI\vendor\htmlawed\htmlawed)
phpmailer/phpmailer version 6.6.0 in (E:\GLPI\vendor\phpmailer\phpmailer\src)
simplepie/simplepie version 1.5.8 in (E:\GLPI\vendor\simplepie\simplepie\library)
tecnickcom/tcpdf version 6.4.4 in (E:\GLPI\vendor\tecnickcom\tcpdf)
michelf/php-markdown in (E:\GLPI\vendor\michelf\php-markdown\Michelf)
true/punycode in (E:\GLPI\vendor\true\punycode\src)
iamcal/lib_autolink in (E:\GLPI\vendor\iamcal\lib_autolink)
sabre/dav in (E:\GLPI\vendor\sabre\dav\lib\DAV)
sabre/http in (E:\GLPI\vendor\sabre\http\lib)
sabre/uri in (E:\GLPI\vendor\sabre\uri\lib)
sabre/vobject in (E:\GLPI\vendor\sabre\vobject\lib)
laminas/laminas-i18n in (E:\GLPI\vendor\laminas\laminas-i18n\src)
laminas/laminas-servicemanager in (E:\GLPI\vendor\laminas\laminas-servicemanager\src)
monolog/monolog in (E:\GLPI\vendor\monolog\monolog\src\Monolog)
sebastian/diff in (E:\GLPI\vendor\sebastian\diff\src)
donatj/phpuseragentparser in (E:\GLPI\vendor\donatj\phpuseragentparser\src\UserAgent)
elvanto/litemoji in (E:\GLPI\vendor\elvanto\litemoji\src)
symfony/console in (E:\GLPI\vendor\symfony\console)
scssphp/scssphp in (E:\GLPI\vendor\scssphp\scssphp\src)
laminas/laminas-mail in (E:\GLPI\vendor\laminas\laminas-mail\src\Protocol)
laminas/laminas-mime in (E:\GLPI\vendor\laminas\laminas-mime\src)
rlanvin/php-rrule in (E:\GLPI\vendor\rlanvin\php-rrule\src)
blueimp/jquery-file-upload in (E:\GLPI\vendor\blueimp\jquery-file-upload\server\php)
ramsey/uuid in (E:\GLPI\vendor\ramsey\uuid\src)
psr/log in (E:\GLPI\vendor\psr\log\Psr\Log)
psr/simple-cache in (E:\GLPI\vendor\psr\simple-cache\src)
psr/cache in (E:\GLPI\vendor\psr\cache\src)
league/csv in (E:\GLPI\vendor\league\csv\src)
mexitek/phpcolors in (E:\GLPI\vendor\mexitek\phpcolors\src\Mexitek\PHPColors)
guzzlehttp/guzzle in (E:\GLPI\vendor\guzzlehttp\guzzle\src)
guzzlehttp/psr7 in (E:\GLPI\vendor\guzzlehttp\psr7\src)
glpi-project/inventory_format in (E:\GLPI\vendor\glpi-project\inventory_format\lib\php)
wapmorgan/unified-archive in (E:\GLPI\vendor\wapmorgan\unified-archive\src)
paragonie/sodium_compat in (E:\GLPI\vendor\paragonie\sodium_compat\src)
symfony/cache in (E:\GLPI\vendor\symfony\cache)
html2text/html2text in (E:\GLPI\vendor\html2text\html2text\src)
symfony/css-selector in (E:\GLPI\vendor\symfony\css-selector)
symfony/dom-crawler in (E:\GLPI\vendor\symfony\dom-crawler)
twig/twig in (E:\GLPI\vendor\twig\twig\src)
twig/string-extra in (E:\GLPI\vendor\twig\string-extra)
symfony/polyfill-ctype not found
symfony/polyfill-iconv not found
symfony/polyfill-mbstring not found
symfony/polyfill-php80 not found
symfony/polyfill-php81 not found

LDAP directories

 
Server: 'ldap://srvadsel01', Port: '389', BaseDN: 'DC=spurgin,DC=local', Connection filter: none, RootDN: 'CN=glpisvc,OU=Compte
        Service,OU=Z_GENERIQUE,DC=spurgin,DC=local', Use TLS: none

SQL replicas

 
Not active

Notifications

 
Way of sending emails: SMTP (anonymous@192.168.28.237)

Plugins list

 
    glpiinventory        Name: GLPI Inventory                 Version: 1.0.0      State: Enabled

Anything else?

After some investigation it looks like, the same device is detected on multiple computer. Probably because the device has a generic serial number. This issue seems to be related from using docks/hub or even switches. Everytime a computer perform an inventory with the bugged/dupplicated device, the device move from one computer to another, creating logs entries. For example my network card adapter is a Realtek on a HP dock, but it has the same serial number on all computers.

[steph461]

To avoid a lot of logs, I purged the logs, I had to keep it for 1 year but I set it to 6 months

Log before changement over 6 month

after :

[steph461]

I have same issue, this generates too many logs over a long period of time

[SteadEXE]

To avoid a lot of logs, I purged the logs, I had to keep it for 1 year but I set it to 6 months

Log before changement over 6 month

after :

I did the same to prevent bloating the database.

[SteadEXE]

I have a device USB Device #573 that get removed all the time, I went on the device itself, it has 65000 lines of history, everytime it moves from one computer to another. It changes manufacturer all the time too.

[SteadEXE]

I have same issue, this generates too many logs over a long period of time

I am pretty sure if you go on this device 1270 you will get a lot of history for it and I am sure this device is used by a lot of computers.

[SteadEXE]

Here is an example of a device moving all the day:

[steph461]

@SteadEXE

device 1270, same issue ping pong on two computer

[steph461]

But I don't have this problem on all computers. For the moment , I found two computer that have this bug

[steph461]

I try delete USB Composite Device (1270)

Scan agent glpi my computer , create USB Composite Device (1846) Scan the second computer , USB Composite Device (1846).

maybe delete the two computer and then new create with agent GLPI, i wil try. My data is a migration 9.5.7 ==> 10.0.0 ( april 2022)

[SteadEXE]

On my side, I guess this is because we use dock/usb hub so a lot of computers detect devices with same serial number. I don't know if somebody from GLPI team has an idea of how can we mitigate the issue in this case

[trasher]

Could you please provide anonymized inventory files that permit to reproduce this issue?

[SteadEXE]

inventory files.zip

Those are two computers (my collegue's one and mine).

We apparently share RTL8153 Gigabit Ethernet Adapter and USB Composite Devices.

[SteadEXE]

I guess a fix would be to allow a device to be used by more than one computer. For real devices it will not change anything. And for dupplicated serial number it will stop the ping pong. Another fix would be to use a combo agent+serial number, but it prevents moving a device from one computer to another which is not ideal for stock management.

[trasher]

Thanks for inventory files, I'll take a look next week

[trasher]

I've inspected and tested provided inventory files; and it seems there is nothing we can really do; those peripherals are strictly the same on both computers.

There is an option in the peripheral form called "Management type" you can set to "Global management" to prevent that kind of case. You have to inventory the first computer, set "dumb" peripherals to global management and then import all other computers. Doing that, all linked peripherals stay on first computer, no entries in log (all is OK on that side), but as far as I've seen, those ones are never linked at all on second computer (which is maybe a bug).

Could you please test?

[SteadEXE]

I did it on "dump" devices. Do you think it is possible to add a crontask to automatically detect those devices and change their management mode to global management if it moves more than x times over y days ? So people who want to keep their database clean can enable the crontask. Maybe it could be a fix.

I guess without changing anything it can uses "link created" entries of history.

I am still new with GLPI source code.

[SteadEXE]

Update: With global management it keeps changing the manufacturer of these devices and some infos. It's a really hard task to fix everything by hand on a huge park of computers

[trasher]

could you please provide same inventories in xml format?

[trasher]

I just made a test on glpi9.5 with the fusioninventory plugin; and the behavior is exactly the same.

[trasher]

pull request #11915 improves default blacklist on serials; this exclude several entries

[steph461]

Hello

I try, Its ok with this fix. Thanks @trasher

[SteadEXE]

By the way there is something weird happening, I set global management, I locked the manufacturer field but it keeps trying to update it:

[steph461]

Ah yes in the log USB composite DEVICE

[trasher]

We made several changes you can test with latest nightly:

• remove of some extra logs during inventory
• improve bad serial detection
• rely on existing configuration (Setup > General > Assets) to set global management on peripherals, monitors and printers
• add options in inventory configuration to prevent import of peripherals, monitors and printers

[SteadEXE]

It may be related but when I global management I got my php-errors spammed by:

[2022-07-01 01:57:42] glpiphplog.WARNING: *** PHP User Warning (512): getFromDBByCrit expects to get one result, 11 found in query "SELECT id FROM glpi_computers_items WHERE itemtype = 'Peripheral' AND items_id = '3'". in E:\GLPI\src\CommonDBTM.php at line 396 Backtrace : src\CommonDBTM.php:396 trigger_error() src\Inventory\Asset\InventoryAsset.php:361 CommonDBTM->getFromDBByCrit() src\Inventory\Asset\Peripheral.php:250 Glpi\Inventory\Asset\InventoryAsset->addOrMoveItem() src\Inventory\Asset\MainAsset.php:811 Glpi\Inventory\Asset\Peripheral->handle() src\Inventory\Asset\MainAsset.php:731 Glpi\Inventory\Asset\MainAsset->handleAssets() src\RuleImportAsset.php:945 Glpi\Inventory\Asset\MainAsset->rulepassed() src\Rule.php:1510 RuleImportAsset->executeActions() src\RuleCollection.php:1591 Rule->process() src\Inventory\Asset\MainAsset.php:550 RuleCollection->processAllRules() src\Inventory\Inventory.php:697 Glpi\Inventory\Asset\MainAsset->handle() src\Inventory\Inventory.php:331 Glpi\Inventory\Inventory->handleItem() src\Inventory\Request.php:360 Glpi\Inventory\Inventory->doInventory() src\Inventory\Request.php:90 Glpi\Inventory\Request->inventory() src\Agent\Communication\AbstractRequest.php:325 Glpi\Inventory\Request->handleAction() src\Agent\Communication\AbstractRequest.php:237 Glpi\Agent\Communication\AbstractRequest->handleJSONRequest() front\inventory.php:86 Glpi\Agent\Communication\AbstractRequest->handleRequest() plugins\glpiinventory\front\communication.php:72 include_once() plugins\glpiinventory\index.php:45 include_once()

The 11 results are the 11 computers sharing the same global device.

[cedric-anne]

It may be related but when I global management I got my php-errors spammed by:

[2022-07-01 01:57:42] glpiphplog.WARNING: *** PHP User Warning (512): getFromDBByCrit expects to get one result, 11 found in query "SELECT id FROM glpi_computers_items WHERE itemtype = 'Peripheral' AND items_id = '3'". in E:\GLPI\src\CommonDBTM.php at line 396 Backtrace : src\CommonDBTM.php:396 trigger_error() src\Inventory\Asset\InventoryAsset.php:361 CommonDBTM->getFromDBByCrit() src\Inventory\Asset\Peripheral.php:250 Glpi\Inventory\Asset\InventoryAsset->addOrMoveItem() src\Inventory\Asset\MainAsset.php:811 Glpi\Inventory\Asset\Peripheral->handle() src\Inventory\Asset\MainAsset.php:731 Glpi\Inventory\Asset\MainAsset->handleAssets() src\RuleImportAsset.php:945 Glpi\Inventory\Asset\MainAsset->rulepassed() src\Rule.php:1510 RuleImportAsset->executeActions() src\RuleCollection.php:1591 Rule->process() src\Inventory\Asset\MainAsset.php:550 RuleCollection->processAllRules() src\Inventory\Inventory.php:697 Glpi\Inventory\Asset\MainAsset->handle() src\Inventory\Inventory.php:331 Glpi\Inventory\Inventory->handleItem() src\Inventory\Request.php:360 Glpi\Inventory\Inventory->doInventory() src\Inventory\Request.php:90 Glpi\Inventory\Request->inventory() src\Agent\Communication\AbstractRequest.php:325 Glpi\Inventory\Request->handleAction() src\Agent\Communication\AbstractRequest.php:237 Glpi\Agent\Communication\AbstractRequest->handleJSONRequest() front\inventory.php:86 Glpi\Agent\Communication\AbstractRequest->handleRequest() plugins\glpiinventory\front\communication.php:72 include_once() plugins\glpiinventory\index.php:45 include_once()

The 11 results are the 11 computers sharing the same global device.

Please open a new issue. Multiple changes have been made on this part, and it would probably better to consider this as another bug and start its analysis from scratch.

[SteadEXE]

Sure I'll do it.