Special Character not well encoded in inventory software name

[lexcyr]

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

Is there an existing issue for this?

• [X] I have searched the existing issues

Version

10.0.10

Bug description

After inventory, the same software shows with correct characters on one device and with wrong character to another. e.g: Aperçu de code-barres Windows Device1

Device2

When i check the import json file the software name is :

 {
                "arch": "neutral",
                "comments": "Bo\u00eete de dialogue Aper\u00e7u de code-barres Windows",
                "folder": "C:\\Windows\\SystemApps\\Windows.CBSPreview_cw5n1h2txyewy",
                "from": "uwp",
                "install_date": "2019-12-07",
                "name": "Aper\u00e7u de code-barres Windows",
                "publisher": "Microsoft Corporation",
                "system_category": "system",
                "version": "10.0.18362.449"
            },

We have the same information in the json of both devices.

Why the encoding works on one and not to the second one. I have the problème on the Device 1 on all software with special characters.

Relevant log output

No response

Page URL

No response

Steps To reproduce

The issue is random on differents devices.

Your GLPI setup information

Information about system installation and configuration

GLPI 10.0.10 ( => /var/www/glpi)
Installation mode: TARBALL
Current language:en_GB

Server

 
Operating system: Linux glpi 5.14.0-284.25.1.el9_2.x86_64 #​1 SMP PREEMPT_DYNAMIC Wed Aug 2 14:53:30
    UTC 2023 x86_64
PHP 8.2.10 fpm-fcgi (Core, PDO, Phar, Reflection, SPL, SimpleXML, Zend OPcache, apcu, bz2, calendar, cgi-fcgi, ctype, curl,
    date, dom, exif, fileinfo, filter, ftp, gd, gettext, hash, iconv, imap, intl, json, ldap, libxml, mbstring, mysql, mysqli,
    mysqlnd, openssl, pcre, pdo_mysql, pdo_sqlite, pdo_sqlsrv, random, session, sockets, sodium, sqlite3, sqlsrv, standard,
    tokenizer, xml, xmlreader, xmlrpc, xmlwriter, xsl, zip, zlib)
Setup: max_execution_time="600" memory_limit="512M" post_max_size="8M" safe_mode="" session.save_handler="files"
    upload_max_filesize="2M" 
Software: Apache/2.4.53 (Rocky Linux) OpenSSL/3.0.7 ()
    Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36 Edg/118.0.2088.61
Server Software: MariaDB Server
    Server Version: 10.5.16-MariaDB
    Server SQL Mode: STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
    Parameters: xxxxxxx
    Host info xxxxxxx

PHP version (8.2.10) is supported.
Sessions configuration is OK.
Allocated memory is sufficient.
mysqli extension is installed.
Following extensions are installed: dom, fileinfo, filter, libxml, json, simplexml, xmlreader, xmlwriter.
curl extension is installed.
gd extension is installed.
intl extension is installed.
zlib extension is installed.
The constant SODIUM_CRYPTO_AEAD_XCHACHA20POLY1305_IETF_NPUBBYTES is present.
Database engine version (10.5.16) is supported.
No files from previous GLPI version detected.
The log file has been created successfully.
Write access to /var/www/glpi/files/_cache has been validated.
Write access to /var/www/glpi/config has been validated.
Write access to /var/www/glpi/files/_cron has been validated.
Write access to /var/www/glpi/files has been validated.
Write access to /var/www/glpi/files/_dumps has been validated.
Write access to /var/www/glpi/files/_graphs has been validated.
Write access to /var/www/glpi/files/_lock has been validated.
Write access to /var/www/glpi/files/_pictures has been validated.
Write access to /var/www/glpi/files/_plugins has been validated.
Write access to /var/www/glpi/files/_rss has been validated.
Write access to /var/www/glpi/files/_sessions has been validated.
Write access to /var/www/glpi/files/_tmp has been validated.
Write access to /var/www/glpi/files/_uploads has been validated.
SELinux configuration is OK.

Web server root directory configuration seems safe.
Sessions configuration is secured.
OS and PHP are relying on 64 bits integers.
exif extension is installed.
ldap extension is installed.
openssl extension is installed.
Following extensions are installed: bz2, Phar, zip.
Zend OPcache extension is installed.
Following extensions are installed: ctype, iconv, mbstring, sodium.
Write access to /var/www/glpi/marketplace has been validated.
Access to timezone database (mysql) is not allowed.

GLPI constants

 
GLPI_ROOT: "/var/www/glpi"
GLPI_CONFIG_DIR: "/var/www/glpi/config"
GLPI_VAR_DIR: "/var/www/glpi/files"
GLPI_MARKETPLACE_DIR: "/var/www/glpi/marketplace"
GLPI_USE_CSRF_CHECK: "1"
GLPI_CSRF_EXPIRES: "7200"
GLPI_CSRF_MAX_TOKENS: "100"
GLPI_USE_IDOR_CHECK: "1"
GLPI_IDOR_EXPIRES: "7200"
GLPI_ALLOW_IFRAME_IN_RICH_TEXT: false
GLPI_SERVERSIDE_URL_ALLOWLIST: ["/^(https?|feed):\\/\\/[^@:]+(\\/.*)?$/"]
GLPI_TELEMETRY_URI: "https://telemetry.glpi-project.org"
GLPI_INSTALL_MODE: "TARBALL"
GLPI_NETWORK_MAIL: "glpi@teclib.com"
GLPI_NETWORK_SERVICES: "https://services.glpi-network.com"
GLPI_MARKETPLACE_ALLOW_OVERRIDE: true
GLPI_MARKETPLACE_MANUAL_DOWNLOADS: true
GLPI_USER_AGENT_EXTRA_COMMENTS: ""
GLPI_DISABLE_ONLY_FULL_GROUP_BY_SQL_MODE: "1"
GLPI_AJAX_DASHBOARD: "1"
GLPI_CALDAV_IMPORT_STATE: 0
GLPI_DEMO_MODE: "0"
GLPI_CENTRAL_WARNINGS: "1"
GLPI_DOC_DIR: "/var/www/glpi/files"
GLPI_CACHE_DIR: "/var/www/glpi/files/_cache"
GLPI_CRON_DIR: "/var/www/glpi/files/_cron"
GLPI_DUMP_DIR: "/var/www/glpi/files/_dumps"
GLPI_GRAPH_DIR: "/var/www/glpi/files/_graphs"
GLPI_LOCAL_I18N_DIR: "/var/www/glpi/files/_locales"
GLPI_LOCK_DIR: "/var/www/glpi/files/_lock"
GLPI_LOG_DIR: "/var/www/glpi/files/_log"
GLPI_PICTURE_DIR: "/var/www/glpi/files/_pictures"
GLPI_PLUGIN_DOC_DIR: "/var/www/glpi/files/_plugins"
GLPI_RSS_DIR: "/var/www/glpi/files/_rss"
GLPI_SESSION_DIR: "/var/www/glpi/files/_sessions"
GLPI_TMP_DIR: "/var/www/glpi/files/_tmp"
GLPI_UPLOAD_DIR: "/var/www/glpi/files/_uploads"
GLPI_INVENTORY_DIR: "/var/www/glpi/files/_inventories"
GLPI_NETWORK_REGISTRATION_API_URL: "https://services.glpi-network.com/api/registration/"
GLPI_MARKETPLACE_PLUGINS_API_URI: "https://services.glpi-network.com/api/marketplace/"
GLPI_I18N_DIR: "/var/www/glpi/locales"
GLPI_VERSION: "10.0.10"
GLPI_SCHEMA_VERSION: "10.0.10@05de68add675fb55abaeec10f3a2552085594a16"
GLPI_MARKETPLACE_PRERELEASES: false
GLPI_MIN_PHP: "7.4.0"
GLPI_MAX_PHP: "8.4.0"
GLPI_YEAR: "2023"

Libraries

 
htmlawed/htmlawed version 1.2.14 in (/var/www/glpi/vendor/htmlawed/htmlawed)
phpmailer/phpmailer version 6.8.0 in (/var/www/glpi/vendor/phpmailer/phpmailer/src)
simplepie/simplepie version 1.5.8 in (/var/www/glpi/vendor/simplepie/simplepie/library)
tecnickcom/tcpdf version 6.6.2 in (/var/www/glpi/vendor/tecnickcom/tcpdf)
michelf/php-markdown in (/var/www/glpi/vendor/michelf/php-markdown/Michelf)
true/punycode in (/var/www/glpi/vendor/true/punycode/src)
iamcal/lib_autolink in (/var/www/glpi/vendor/iamcal/lib_autolink)
sabre/dav in (/var/www/glpi/vendor/sabre/dav/lib/DAV)
sabre/http in (/var/www/glpi/vendor/sabre/http/lib)
sabre/uri in (/var/www/glpi/vendor/sabre/uri/lib)
sabre/vobject in (/var/www/glpi/vendor/sabre/vobject/lib)
laminas/laminas-i18n in (/var/www/glpi/vendor/laminas/laminas-i18n/src)
laminas/laminas-servicemanager in (/var/www/glpi/vendor/laminas/laminas-servicemanager/src)
monolog/monolog in (/var/www/glpi/vendor/monolog/monolog/src/Monolog)
sebastian/diff in (/var/www/glpi/vendor/sebastian/diff/src)
donatj/phpuseragentparser in (/var/www/glpi/vendor/donatj/phpuseragentparser/src/UserAgent)
elvanto/litemoji in (/var/www/glpi/vendor/elvanto/litemoji/src)
symfony/console in (/var/www/glpi/vendor/symfony/console)
scssphp/scssphp in (/var/www/glpi/vendor/scssphp/scssphp/src)
laminas/laminas-mail in (/var/www/glpi/vendor/laminas/laminas-mail/src/Protocol)
laminas/laminas-mime in (/var/www/glpi/vendor/laminas/laminas-mime/src)
rlanvin/php-rrule in (/var/www/glpi/vendor/rlanvin/php-rrule/src)
ramsey/uuid in (/var/www/glpi/vendor/ramsey/uuid/src)
psr/log in (/var/www/glpi/vendor/psr/log/Psr/Log)
psr/simple-cache in (/var/www/glpi/vendor/psr/simple-cache/src)
psr/cache in (/var/www/glpi/vendor/psr/cache/src)
league/csv in (/var/www/glpi/vendor/league/csv/src)
mexitek/phpcolors in (/var/www/glpi/vendor/mexitek/phpcolors/src/Mexitek/PHPColors)
guzzlehttp/guzzle in (/var/www/glpi/vendor/guzzlehttp/guzzle/src)
guzzlehttp/psr7 in (/var/www/glpi/vendor/guzzlehttp/psr7/src)
glpi-project/inventory_format in (/var/www/glpi/vendor/glpi-project/inventory_format/lib/php)
wapmorgan/unified-archive in (/var/www/glpi/vendor/wapmorgan/unified-archive/src)
paragonie/sodium_compat in (/var/www/glpi/vendor/paragonie/sodium_compat/src)
symfony/cache in (/var/www/glpi/vendor/symfony/cache)
html2text/html2text in (/var/www/glpi/vendor/html2text/html2text/src)
symfony/css-selector in (/var/www/glpi/vendor/symfony/css-selector)
symfony/dom-crawler in (/var/www/glpi/vendor/symfony/dom-crawler)
twig/twig in (/var/www/glpi/vendor/twig/twig/src)
twig/string-extra in (/var/www/glpi/vendor/twig/string-extra)
symfony/polyfill-ctype not found
symfony/polyfill-iconv not found
symfony/polyfill-mbstring not found
symfony/polyfill-php80 not found
symfony/polyfill-php81 not found
symfony/polyfill-php82 in (/var/www/glpi/vendor/symfony/polyfill-php82)
league/oauth2-client in (/var/www/glpi/vendor/league/oauth2-client/src/Provider)
league/oauth2-google in (/var/www/glpi/vendor/league/oauth2-google/src/Provider)
thenetworg/oauth2-azure in (/var/www/glpi/vendor/thenetworg/oauth2-azure/src/Provider)

LDAP directories

 
Server: 'xxxxxxxxx', Port: '389', BaseDN: xxxxxxxxxx', Connection filter:
        '(&(objectClass=user)(objectCategory=person)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))', RootDN:
        'xxxxxxxxxI', Use TLS: none

SQL replicas

 
Not active

Notifications

 
Way of sending emails: PHP

Plugins list

 
    fields               Name: Additional fields              Version: 1.21.6     State: Enabled                                 
        Install Method: Manual
    glpiinventory        Name: GLPI Inventory                 Version: 1.3.1      State: Enabled                                 
        Install Method: Manual
    sccm                 Name: Interface - SCCM               Version: 2.4.3      State: Installed / not activated               
        Install Method: Manual
    manufacturersimports Name: Suppliers imports              Version: 3.0.5      State: Enabled                                 
        Install Method: Manual

Anything else?

GLPI-Agent_v1.5

[trasher]

Please provide an inventory file to reproduce from a fresh GLPI install.

[lexcyr]

Please provide an inventory file to reproduce from a fresh GLPI install. do you mean the json file i can extfact from the glpi asset page ?

[stonebuzz]

Yes

here :

[lexcyr]

Here we go: computer_0_302 (1).json

[trasher]

I do not have any encoding issue using provided file.

[stonebuzz]

Same for me

Can you retry by deeteing related software "Aperçu de code-barres Windows"

and re-run inventory

[lexcyr]

i've completely delated the related software "Aperçu de code-barres Windows" and re-run the inventory on 4 computers. In one of them it was fixed, and for the reste it came back once more with the incorrect character. This issue seems to be really random, and i was thinking about something in the client computer, but after comparing the json file, i wasn't able to find a real difference.

What can cause the server to interpret the same information differently ?

[stonebuzz]

Maybe another software

For "the reste it came back once more with the incorrect character"

Can you post here related inventory file ?

[lexcyr]

45.json 787.json 8963.json

[trasher]

I still do not reproduce with any of those files; I have no idea what can goes wrong on your side. This can be related to a rule, or a server configuration, or maybe an agent configuration.

Try if you can reproduce on a blank GLPI nightly installation (with no rules, no plugin).

[keguira]

same issue here with a 10.0.10 / agent 1.5 (not a fresh install).

I've tried to delete the software and rerun an inventory, the soft keep being created with artifacts.

I have to anonymise my inventory file before uploading it here. I'll also try to have a look into the database and do more digging.

[stonebuzz]

Can you execute this SQL query and past here the result?

SHOW CREATE TABLE glpi_softwares;
SELECT DEFAULT_CHARACTER_SET_NAME, DEFAULT_COLLATION_NAME FROM INFORMATION_SCHEMA.SCHEMATA WHERE SCHEMA_NAME = '10bugfixes';

Here are my results

MariaDB [10bugfixes]> SHOW CREATE TABLE glpi_softwares;
+----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table          | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
+----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| glpi_softwares | CREATE TABLE `glpi_softwares` (
  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `entities_id` int(10) unsigned NOT NULL DEFAULT 0,
  `is_recursive` tinyint(4) NOT NULL DEFAULT 0,
  `name` varchar(255) DEFAULT NULL,
  `comment` text DEFAULT NULL,
  `locations_id` int(10) unsigned NOT NULL DEFAULT 0,
  `users_id_tech` int(10) unsigned NOT NULL DEFAULT 0,
  `groups_id_tech` int(10) unsigned NOT NULL DEFAULT 0,
  `is_update` tinyint(4) NOT NULL DEFAULT 0,
  `softwares_id` int(10) unsigned NOT NULL DEFAULT 0,
  `manufacturers_id` int(10) unsigned NOT NULL DEFAULT 0,
  `is_deleted` tinyint(4) NOT NULL DEFAULT 0,
  `is_template` tinyint(4) NOT NULL DEFAULT 0,
  `template_name` varchar(255) DEFAULT NULL,
  `date_mod` timestamp NULL DEFAULT NULL,
  `users_id` int(10) unsigned NOT NULL DEFAULT 0,
  `groups_id` int(10) unsigned NOT NULL DEFAULT 0,
  `ticket_tco` decimal(20,4) DEFAULT 0.0000,
  `is_helpdesk_visible` tinyint(4) NOT NULL DEFAULT 1,
  `softwarecategories_id` int(10) unsigned NOT NULL DEFAULT 0,
  `is_valid` tinyint(4) NOT NULL DEFAULT 1,
  `date_creation` timestamp NULL DEFAULT NULL,
  `pictures` text DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `date_mod` (`date_mod`),
  KEY `name` (`name`),
  KEY `is_template` (`is_template`),
  KEY `is_update` (`is_update`),
  KEY `softwarecategories_id` (`softwarecategories_id`),
  KEY `entities_id` (`entities_id`),
  KEY `is_recursive` (`is_recursive`),
  KEY `manufacturers_id` (`manufacturers_id`),
  KEY `groups_id` (`groups_id`),
  KEY `users_id` (`users_id`),
  KEY `locations_id` (`locations_id`),
  KEY `users_id_tech` (`users_id_tech`),
  KEY `softwares_id` (`softwares_id`),
  KEY `is_deleted` (`is_deleted`),
  KEY `is_helpdesk_visible` (`is_helpdesk_visible`),
  KEY `groups_id_tech` (`groups_id_tech`),
  KEY `date_creation` (`date_creation`)
) ENGINE=InnoDB AUTO_INCREMENT=2849 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci ROW_FORMAT=DYNAMIC |
+----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0,000 sec)

MariaDB [10bugfixes]> SELECT DEFAULT_CHARACTER_SET_NAME, DEFAULT_COLLATION_NAME FROM INFORMATION_SCHEMA.SCHEMATA WHERE SCHEMA_NAME = '10bugfixes';
+----------------------------+------------------------+
| DEFAULT_CHARACTER_SET_NAME | DEFAULT_COLLATION_NAME |
+----------------------------+------------------------+
| utf8mb4                    | utf8mb4_general_ci     |
+----------------------------+------------------------+
1 row in set (0,001 sec)

[keguira]

here is :

mysql> SHOW CREATE TABLE glpi_softwares;
+----------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table          | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
+----------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| glpi_softwares | CREATE TABLE `glpi_softwares` (
  `id` int unsigned NOT NULL AUTO_INCREMENT,
  `entities_id` int unsigned NOT NULL DEFAULT '0',
  `is_recursive` tinyint NOT NULL DEFAULT '0',
  `name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `comment` text COLLATE utf8mb4_unicode_ci,
  `locations_id` int unsigned NOT NULL DEFAULT '0',
  `users_id_tech` int unsigned NOT NULL DEFAULT '0',
  `groups_id_tech` int unsigned NOT NULL DEFAULT '0',
  `is_update` tinyint NOT NULL DEFAULT '0',
  `softwares_id` int unsigned NOT NULL DEFAULT '0',
  `manufacturers_id` int unsigned NOT NULL DEFAULT '0',
  `is_deleted` tinyint NOT NULL DEFAULT '0',
  `is_template` tinyint NOT NULL DEFAULT '0',
  `template_name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `date_mod` timestamp NULL DEFAULT NULL,
  `users_id` int unsigned NOT NULL DEFAULT '0',
  `groups_id` int unsigned NOT NULL DEFAULT '0',
  `ticket_tco` decimal(20,4) DEFAULT '0.0000',
  `is_helpdesk_visible` tinyint NOT NULL DEFAULT '1',
  `softwarecategories_id` int unsigned NOT NULL DEFAULT '0',
  `is_valid` tinyint NOT NULL DEFAULT '1',
  `date_creation` timestamp NULL DEFAULT NULL,
  `pictures` text COLLATE utf8mb4_unicode_ci,
  PRIMARY KEY (`id`),
  KEY `date_mod` (`date_mod`),
  KEY `name` (`name`),
  KEY `is_template` (`is_template`),
  KEY `is_update` (`is_update`),
  KEY `softwarecategories_id` (`softwarecategories_id`),
  KEY `entities_id` (`entities_id`),
  KEY `is_recursive` (`is_recursive`),
  KEY `manufacturers_id` (`manufacturers_id`),
  KEY `groups_id` (`groups_id`),
  KEY `users_id` (`users_id`),
  KEY `locations_id` (`locations_id`),
  KEY `users_id_tech` (`users_id_tech`),
  KEY `softwares_id` (`softwares_id`),
  KEY `is_deleted` (`is_deleted`),
  KEY `is_helpdesk_visible` (`is_helpdesk_visible`),
  KEY `groups_id_tech` (`groups_id_tech`),
  KEY `date_creation` (`date_creation`)
) ENGINE=InnoDB AUTO_INCREMENT=28444 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci ROW_FORMAT=DYNAMIC |
+----------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql> SELECT DEFAULT_CHARACTER_SET_NAME, DEFAULT_COLLATION_NAME FROM INFORMATION_SCHEMA.SCHEMATA WHERE SCHEMA_NAME = 'glpi';
+----------------------------+------------------------+
| DEFAULT_CHARACTER_SET_NAME | DEFAULT_COLLATION_NAME |
+----------------------------+------------------------+
| utf8mb4                    | utf8mb4_0900_ai_ci     |
+----------------------------+------------------------+
1 row in set (0.00 sec)

I do not have the same DEFAULT_COLLATION_NAME I'm on mysql 8.0.x

[stonebuzz]

Maybe its the Root cause, Can you fix on a test envirronment and retry ?

[lexcyr]

on my side i have more differences:

MariaDB [glpidb]> SHOW CREATE TABLE glpi_softwares;
+----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table          | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
+----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| glpi_softwares | CREATE TABLE `glpi_softwares` (
  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `entities_id` int(10) unsigned NOT NULL DEFAULT 0,
  `is_recursive` tinyint(4) NOT NULL DEFAULT 0,
  `name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `comment` text COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `locations_id` int(10) unsigned NOT NULL DEFAULT 0,
  `users_id_tech` int(10) unsigned NOT NULL DEFAULT 0,
  `groups_id_tech` int(10) unsigned NOT NULL DEFAULT 0,
  `is_update` tinyint(4) NOT NULL DEFAULT 0,
  `softwares_id` int(10) unsigned NOT NULL DEFAULT 0,
  `manufacturers_id` int(10) unsigned NOT NULL DEFAULT 0,
  `is_deleted` tinyint(4) NOT NULL DEFAULT 0,
  `is_template` tinyint(4) NOT NULL DEFAULT 0,
  `template_name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `date_mod` timestamp NULL DEFAULT NULL,
  `users_id` int(10) unsigned NOT NULL DEFAULT 0,
  `groups_id` int(10) unsigned NOT NULL DEFAULT 0,
  `ticket_tco` decimal(20,4) DEFAULT 0.0000,
  `is_helpdesk_visible` tinyint(4) NOT NULL DEFAULT 1,
  `softwarecategories_id` int(10) unsigned NOT NULL DEFAULT 0,
  `is_valid` tinyint(4) NOT NULL DEFAULT 1,
  `date_creation` timestamp NULL DEFAULT NULL,
  `pictures` text COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `date_mod` (`date_mod`),
  KEY `name` (`name`),
  KEY `is_template` (`is_template`),
  KEY `is_update` (`is_update`),
  KEY `softwarecategories_id` (`softwarecategories_id`),
  KEY `entities_id` (`entities_id`),
  KEY `is_recursive` (`is_recursive`),
  KEY `manufacturers_id` (`manufacturers_id`),
  KEY `groups_id` (`groups_id`),
  KEY `users_id` (`users_id`),
  KEY `locations_id` (`locations_id`),
  KEY `users_id_tech` (`users_id_tech`),
  KEY `softwares_id` (`softwares_id`),
  KEY `is_deleted` (`is_deleted`),
  KEY `is_helpdesk_visible` (`is_helpdesk_visible`),
  KEY `groups_id_tech` (`groups_id_tech`),
  KEY `date_creation` (`date_creation`)
) ENGINE=InnoDB AUTO_INCREMENT=9659 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci ROW_FORMAT=DYNAMIC |
+----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.000 sec)

MariaDB [glpidb]> SELECT DEFAULT_CHARACTER_SET_NAME, DEFAULT_COLLATION_NAME FROM INFORMATION_SCHEMA.SCHEMATA WHERE SCHEMA_NAME = '10bugfixes';
Empty set (0.000 sec)

Unfortunately i am complete dummy with db, but i can see there is this encoding utf8mb4_unicode_ci coming here while i can't see it on your result and no DEFAULT_CHARACTER_SET_NAME and DEFAULT_CHARACTER_SET_NAME

[stonebuzz]

@lexcyr

for the last query you need to adapt SCHEMA_NAME value, on my side, glpi database is named "10bugfixes", check your config_db.php file to get your database name

[lexcyr]

So i have a complete different result :)

MariaDB [glpidb]> SELECT DEFAULT_CHARACTER_SET_NAME, DEFAULT_COLLATION_NAME FROM INFORMATION_SCHEMA.SCHEMATA WHERE SCHEMA_NAME = 'glpidb';
+----------------------------+------------------------+
| DEFAULT_CHARACTER_SET_NAME | DEFAULT_COLLATION_NAME |
+----------------------------+------------------------+
| utf8                       | utf8_bin               |
+----------------------------+------------------------+
1 row in set (0.000 sec)

[stonebuzz]

Unfortunately i am complete dummy with db

Like anyone who isn't a DBA ^^

@lexcyr @keguira

can you past here your config_db.php file (without password field =) )

here is mine

<?php
class DB extends DBmysql {
   public $dbhost = 'localhost';
   public $dbuser = 'xxxxx';
   public $dbpassword = 'xxxxxx';
   public $dbdefault = '10bugfixes';
   public $use_timezones = true;
   public $use_utf8mb4 = true;     //what interests me
   public $allow_myisam = false;
   public $allow_datetime = false;
   public $allow_signed_keys = false;
}

[keguira]

I've tried to reproduce the issue on my test instance. I could not reproduce the error by testing an inventory file. It made me do another test on my production and what I see is that the issue does not happen when you upload manually an inventory file, it's only when the inventory is done by the usual process.

With a forced inventory :

By reuploading the new inventory file just after :

If the process to record into the database is the same in both use-case, it means that the issue is during the decoding process

[stonebuzz]

Can you retry on GLPI fresh install? (to exclude database encoding)

[keguira]

I'll try, it does not occur with all computer (i forgot to mention that) so i have to borrow a computer.

Note on computer it's not working properly : it's the same windows version, same agent version and installation

[stonebuzz]

From manual import or with glpi-injector (To simulate sending the file as the agent does)

i do not reproduce on my side

[lexcyr]

Unfortunately i am complete dummy with db

Like anyone who isn't a DBA ^^

@lexcyr @keguira

can you past here your config_db.php file (without password field =) )

here we go

<?php
class DB extends DBmysql {
   public $dbhost = '127.0.0.1';
   public $dbuser = 'xxxx';
   public $dbpassword = 'xxxxx';
   public $dbdefault = 'glpidb';
   public $use_utf8mb4 = true;
   public $allow_myisam = false;
   public $allow_datetime = false;
   public $allow_signed_keys = false;
}

[trasher]

This looks like a web server encoding issue. Do you hit directly web server hosting GLPI instance, or do you use a proxy or something similar?

[lexcyr]

from my side i hit directely the webserver hosting GLPI, even if some of the client are behind a VPN

[keguira]

from my side i hit directely the webserver hosting GLPI, even if some of the client are behind a VPN

I have the same architecture.

[keguira]

If i can add information, I've found a computer with this behavior :

• Day A : auto-inventory -> remove all software with special character and add with artifact characters
• Day B : auto-inventory -> remove all software with artifact character and add with special characters
• Day C : auto-inventory -> remove all software with special character and add with artifact characters
• Day D : auto-inventory -> remove all software with artifact character and add with special characters

Here is a composition of screenshots for one software :

I'm currently trying to reproduce the behavior by requesting inventories

[stonebuzz]

@keguira can you post here json inventory result (from agent) ?

(not inventory file downloadable from GLPI)

[g-bougard]

@keguira to generate a json, you must open an administrative console on the system, change your directory to the glpi-agent installation folder and run the following command:

glpi-agent --config=none --logger=stderr --debug --json --local=. --force

This will generate the original json final in the installation folder.

You can also share the command run output if this can help.

[keguira]

Yes, I have to reach the user (or another one with this beavior) and anonymise the result. I'll try to get it by the end of the week.

[trasher]

Make sure you do not change file encoding when editing...

[keguira]

Make sure you do not change file encoding when editing...

yes of course

[keguira]

Also,@lexcyr , do you see this bug also in other fields / objects ? I have the issue at random in manufacturer names, software names and operating system names.

[stonebuzz]

@keguira have you tried on a new fresh isntall

for recall, the DEFAULT_COLLATION_NAME of your DB could be at fault

I'm not an expert

+----------------------------+------------------------+ | DEFAULT_CHARACTER_SET_NAME | DEFAULT_COLLATION_NAME | +----------------------------+------------------------+ | utf8mb4 | utf8mb4_0900_ai_ci | +----------------------------+------------------------+

[lexcyr]

Also,@lexcyr , do you see this bug also in other fields / objects ? I have the issue at random in manufacturer names, software names and operating system names.

I have it only on Manufacturer and Software names, for the moment.

[keguira]

add a reaction after you retrieve the file, i'll remove it after

I checked before modification, the file was and is still, UTF-8 without BOM

[keguira]

If i can add information, I've found a computer with this behavior :

• Day A : auto-inventory -> remove all software with special character and add with artifact characters
• Day B : auto-inventory -> remove all software with artifact character and add with special characters
• Day C : auto-inventory -> remove all software with special character and add with artifact characters
• Day D : auto-inventory -> remove all software with artifact character and add with special characters ...

Another context info : our cleansoftware task was launched daily. such as the soft is recreated in base. I changed it to mountly to see if things are sorted out automatically. It may be comming from the process of "check if exists". If I recall correctly, there as been encoding cleaning not so long ago

@keguira have you tried on a new fresh isntall

for recall, the DEFAULT_COLLATION_NAME of your DB could be at fault

I'm not an expert

+----------------------------+------------------------+ | DEFAULT_CHARACTER_SET_NAME | DEFAULT_COLLATION_NAME | +----------------------------+------------------------+ | utf8mb4 | utf8mb4_0900_ai_ci | +----------------------------+------------------------+

will be done by the end of the week

[g-bougard]

add a reaction after you retrieve the file, i'll remove it after

FX052-092.example.org-2022-11-25-09-15-40.json.zip

I checked before modification, the file was and is still, UTF-8 without BOM

I confirm the inventory is well-form and the issue should be server-side... somewhere ^^

[keguira]

Ok, i'm not that crazy, i'll try with a spare to reproduce the issue and add some debug in php or ... i don't know.

For the record, in the past, i saw that chineese characters were not displaye / recorded correctly (only square characters). As I was on other issues in our system, I did not look into it. It's only recently (in the past 3~4 months) that I saw correctly the chinese characters (no modification in the dattabase conf, only glpi upgrades). I cannot give more info about history as we keep only the last 6 month and we did a cleanup as it was crippled due to a now corrected bugs

I use some dictionnary rules but not for operating systems right now. We have to build them with the users next trimesters so it does not seems to come from this process. Or not ? I don't know.

@g-bougard could you remove the link from the quote and remove history from the comment ? Even if it was anonymized ... privacy, corporate, security, etc ...

[g-bougard]

@keguira I fixed the quote.

I noticed you're using php-fpm in the system report: PHP 8.2.10 fpm-fcgi

Can you try with another configuration not using fpm-fcgi ? Like Apache+mod_php ?

[keguira]

Initial bug was not from me so it's not my server config on report. In our compagny, we use PHP 8.1.x with nginx. But yes, it's still PHP fpm-cgi.

It's something usually recommended isn't it ? I missed something ?

I could try an alternative BUT i do not reproduce the bug on test environnent (and i cannot change prod config right now), it's pretty random and not on all computer such i would have to "hunt" users. I can try to add this to our backlog but i'm not confident in this...

[trasher]

Nothing is recommended since that should not really change anything. We already had cases that some configurations were not working as expected; but that was probably related to an underlying configuration.

We do not reproduce; until we're able to get a reproducible use case, there is nothgin we can do. In some past versions, we've identified issues related to encoding or not displayed characters, all cases have been resolved an added to our tests suite. All we can do right now is give possible causes. I use only apache, everything should work the same on nginx... Unless you have a load balancer on your entry point, or use a third party software like a vpn or something similar between agent and glpi instance, I do not have any further idea.

Since you do not reproduce on your own test environment, it seems there is something different with your prod; maybe finding the difference may help to reproduce and fix.

@lexcyr may also take a deeper look on all that, since he has the same issue.

[keguira]

Yes yes, of course. I'm a dev myself and know that we have to be able to reproduce to fix. That's why i'm trying to have a use-case i can manage myself without relying on another user. As our users are on windows I cannot easily change the destination server unless i apply a trick on our production to redirect on our test and it may temper the test. Tomorrow, i'll delete all software with artifact and see what happens. Next week, if there are new "artifacted" ones, i'll delete their counterparts to see what happens. If I identify something, I'll keep you all informed

[github-actions[bot]]

There has been no activity on this issue for some time and therefore it is considered stale and will be closed automatically in 10 days.

If this issue is related to a bug, please try to reproduce on latest release. If the problem persist, feel free to add a comment to revive this issue. If it is related to a new feature, please open a topic to discuss with community about this enhancement on suggestion website.

You may also consider taking a subscription to get professionnal support or contact GLPI editor team directly.