glpi-project / glpi

GLPI is a Free Asset and IT Management Software package, Data center management, ITIL Service Desk, licenses tracking and software auditing.
https://glpi-project.org
GNU General Public License v3.0
4.33k stars 1.3k forks source link

MSG attachment in an email that is imported via the POP3 Connector, the content is changed or converted into an EML file that is no longer fully readable #17464

Open schieffahrer opened 4 months ago

schieffahrer commented 4 months ago

Code of Conduct

Is there an existing issue for this?

Version

10.0.15

Bug description

We sometimes receive MSG files from users sent to the ticket system by email.

If we create a ticket manually and add a MSG file, it is displayed identically. The situation is different with email imports. There are several problems here. The MSG file attachment is converted into an EML file. This is then unusable. For unknown reasons, the MSG is sometimes imported, but due to a change within the file, the display no longer works. German umlauts and special characters are displayed strangely. Is it possible to set GLPI so that MSG files are saved exactly as they are sent?

Examples are: "&nbs=;" or "Datenschutzb=stimmungen" (correct Datenschutzbestimmungen) 2024-07-07 17_10_16-MSG

Relevant log output

No response

Page URL

No response

Steps To reproduce

  1. create an email to the ticket system with Outlook
  2. add another email from Outlook as an attachment (is then automatically a MSG file)
  3. GLPI imports this email and creates a ticket from it with an EML file that was originally the MSG file
  4. open EML file from the ticket system

Your GLPI setup information

Informationen über das System, die Installation und die Konfiguration
GLPI 10.0.15 ( => /var/www/clients/client1/web1/web/glpi)
Installation mode: TARBALL
Current language:de_DE

Server
 
Operating system: Linux webspace 5.10.0-30-amd64 #​1 SMP Debian 5.10.218-1 (2024-06-01) x86_64
PHP 8.2.20 cgi-fcgi (Core, FFI, PDO, Phar, Reflection, SPL, SimpleXML, Zend OPcache, apcu, bcmath, bz2, calendar, cgi-fcgi,
    ctype, curl, date, dom, exif, fileinfo, filter, ftp, gd, gettext, hash, iconv, igbinary, imap, intl, json, ldap, libxml,
    mbstring, mcrypt, msgpack, mysqli, mysqlnd, openssl, pcntl, pcre, pdo_mysql, pdo_sqlite, posix, pspell, random, readline, redis,
    session, shmop, soap, sockets, sodium, sqlite3, standard, sysvmsg, sysvsem, sysvshm, tidy, tokenizer, xml, xmlreader, xmlrpc,
    xmlwriter, xsl, zip, zlib)
Setup: max_execution_time="30" memory_limit="256M" post_max_size="512M" safe_mode="" session.save_handler="files"
    upload_max_filesize="512M" disable_functions="" 
Software: Apache ()
    Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
Server Software: Debian 11
    Server Version: 10.5.23-MariaDB-0+deb11u1
    Server SQL Mode: NO_ENGINE_SUBSTITUTION
    Parameters: c1ticket@localhost/c1ticket
    Host info: Localhost via UNIX socket

PHP version (8.2.20) is supported.
Sessions configuration is OK.
Allocated memory is sufficient.
mysqli extension is installed.
Following extensions are installed: dom, fileinfo, filter, libxml, json, simplexml, xmlreader, xmlwriter.
curl extension is installed.
gd extension is installed.
intl extension is installed.
zlib extension is installed.
The constant SODIUM_CRYPTO_AEAD_XCHACHA20POLY1305_IETF_NPUBBYTES is present.
Database engine version (10.5.23) is supported.
No files from previous GLPI version detected.
The log file has been created successfully.
Write access to /var/www/clients/client1/web1/web/glpi/files/_cache has been validated.
Write access to /var/www/clients/client1/web1/web/glpi/files/_cron has been validated.
Write access to /var/www/clients/client1/web1/web/glpi/files has been validated.
Write access to /var/www/clients/client1/web1/web/glpi/files/_dumps has been validated.
Write access to /var/www/clients/client1/web1/web/glpi/files/_graphs has been validated.
Write access to /var/www/clients/client1/web1/web/glpi/files/_lock has been validated.
Write access to /var/www/clients/client1/web1/web/glpi/files/_pictures has been validated.
Write access to /var/www/clients/client1/web1/web/glpi/files/_plugins has been validated.
Write access to /var/www/clients/client1/web1/web/glpi/files/_rss has been validated.
Write access to /var/www/clients/client1/web1/web/glpi/files/_sessions has been validated.
Write access to /var/www/clients/client1/web1/web/glpi/files/_tmp has been validated.
Write access to /var/www/clients/client1/web1/web/glpi/files/_uploads has been validated.

Web server root directory configuration seems safe.
Sessions configuration is secured.
OS and PHP are relying on 64 bits integers.
exif extension is installed.
ldap extension is installed.
openssl extension is installed.
Following extensions are installed: bz2, Phar, zip.
Zend OPcache extension is installed.
Following extensions are installed: ctype, iconv, mbstring, sodium.
Write access to /var/www/clients/client1/web1/web/glpi/marketplace has been validated.
Timezones seems loaded in database.

GLPI constants
 
GLPI_ROOT: "/var/www/clients/client1/web1/web/glpi"
GLPI_CONFIG_DIR: "/var/www/clients/client1/web1/web/glpi/config"
GLPI_VAR_DIR: "/var/www/clients/client1/web1/web/glpi/files"
GLPI_MARKETPLACE_DIR: "/var/www/clients/client1/web1/web/glpi/marketplace"
GLPI_USE_CSRF_CHECK: "1"
GLPI_CSRF_EXPIRES: "7200"
GLPI_CSRF_MAX_TOKENS: "100"
GLPI_USE_IDOR_CHECK: "1"
GLPI_IDOR_EXPIRES: "7200"
GLPI_ALLOW_IFRAME_IN_RICH_TEXT: false
GLPI_SERVERSIDE_URL_ALLOWLIST: ["/^(https?|feed):\\/\\/[^@:]+(\\/.*)?$/"]
GLPI_TELEMETRY_URI: "https://telemetry.glpi-project.org"
GLPI_INSTALL_MODE: "TARBALL"
GLPI_NETWORK_MAIL: "glpi@teclib.com"
GLPI_NETWORK_SERVICES: "https://services.glpi-network.com"
GLPI_MARKETPLACE_ALLOW_OVERRIDE: true
GLPI_MARKETPLACE_MANUAL_DOWNLOADS: true
GLPI_USER_AGENT_EXTRA_COMMENTS: ""
GLPI_DISABLE_ONLY_FULL_GROUP_BY_SQL_MODE: "1"
GLPI_AJAX_DASHBOARD: "1"
GLPI_CALDAV_IMPORT_STATE: 0
GLPI_DEMO_MODE: "0"
GLPI_CENTRAL_WARNINGS: "1"
GLPI_TEXT_MAXSIZE: "4000"
GLPI_DOC_DIR: "/var/www/clients/client1/web1/web/glpi/files"
GLPI_CACHE_DIR: "/var/www/clients/client1/web1/web/glpi/files/_cache"
GLPI_CRON_DIR: "/var/www/clients/client1/web1/web/glpi/files/_cron"
GLPI_DUMP_DIR: "/var/www/clients/client1/web1/web/glpi/files/_dumps"
GLPI_GRAPH_DIR: "/var/www/clients/client1/web1/web/glpi/files/_graphs"
GLPI_LOCAL_I18N_DIR: "/var/www/clients/client1/web1/web/glpi/files/_locales"
GLPI_LOCK_DIR: "/var/www/clients/client1/web1/web/glpi/files/_lock"
GLPI_LOG_DIR: "/var/www/clients/client1/web1/web/glpi/files/_log"
GLPI_PICTURE_DIR: "/var/www/clients/client1/web1/web/glpi/files/_pictures"
GLPI_PLUGIN_DOC_DIR: "/var/www/clients/client1/web1/web/glpi/files/_plugins"
GLPI_RSS_DIR: "/var/www/clients/client1/web1/web/glpi/files/_rss"
GLPI_SESSION_DIR: "/var/www/clients/client1/web1/web/glpi/files/_sessions"
GLPI_TMP_DIR: "/var/www/clients/client1/web1/web/glpi/files/_tmp"
GLPI_UPLOAD_DIR: "/var/www/clients/client1/web1/web/glpi/files/_uploads"
GLPI_INVENTORY_DIR: "/var/www/clients/client1/web1/web/glpi/files/_inventories"
GLPI_NETWORK_REGISTRATION_API_URL: "https://services.glpi-network.com/api/registration/"
GLPI_MARKETPLACE_PLUGINS_API_URI: "https://services.glpi-network.com/api/marketplace/"
GLPI_I18N_DIR: "/var/www/clients/client1/web1/web/glpi/locales"
GLPI_VERSION: "10.0.15"
GLPI_SCHEMA_VERSION: "10.0.15@2eed74704cb07e0bac48b933cbd5c1c356f09629"
GLPI_MARKETPLACE_PRERELEASES: false
GLPI_MIN_PHP: "7.4.0"
GLPI_MAX_PHP: "8.4.0"
GLPI_YEAR: "2024"

Libraries
 
htmlawed/htmlawed version 1.2.14 in (/var/www/clients/client1/web1/web/glpi/vendor/htmlawed/htmlawed)
phpmailer/phpmailer version 6.8.0 in (/var/www/clients/client1/web1/web/glpi/vendor/phpmailer/phpmailer/src)
simplepie/simplepie version 1.5.8 in (/var/www/clients/client1/web1/web/glpi/vendor/simplepie/simplepie/library)
tecnickcom/tcpdf version 6.7.5 in (/var/www/clients/client1/web1/web/glpi/vendor/tecnickcom/tcpdf)
michelf/php-markdown in (/var/www/clients/client1/web1/web/glpi/vendor/michelf/php-markdown/Michelf)
true/punycode in (/var/www/clients/client1/web1/web/glpi/vendor/true/punycode/src)
iamcal/lib_autolink in (/var/www/clients/client1/web1/web/glpi/vendor/iamcal/lib_autolink)
sabre/dav in (/var/www/clients/client1/web1/web/glpi/vendor/sabre/dav/lib/DAV)
sabre/http in (/var/www/clients/client1/web1/web/glpi/vendor/sabre/http/lib)
sabre/uri in (/var/www/clients/client1/web1/web/glpi/vendor/sabre/uri/lib)
sabre/vobject in (/var/www/clients/client1/web1/web/glpi/vendor/sabre/vobject/lib)
laminas/laminas-i18n in (/var/www/clients/client1/web1/web/glpi/vendor/laminas/laminas-i18n/src)
laminas/laminas-servicemanager in (/var/www/clients/client1/web1/web/glpi/vendor/laminas/laminas-servicemanager/src)
monolog/monolog in (/var/www/clients/client1/web1/web/glpi/vendor/monolog/monolog/src/Monolog)
sebastian/diff in (/var/www/clients/client1/web1/web/glpi/vendor/sebastian/diff/src)
donatj/phpuseragentparser in (/var/www/clients/client1/web1/web/glpi/vendor/donatj/phpuseragentparser/src/UserAgent)
elvanto/litemoji in (/var/www/clients/client1/web1/web/glpi/vendor/elvanto/litemoji/src)
symfony/console in (/var/www/clients/client1/web1/web/glpi/vendor/symfony/console)
scssphp/scssphp in (/var/www/clients/client1/web1/web/glpi/vendor/scssphp/scssphp/src)
laminas/laminas-mail in (/var/www/clients/client1/web1/web/glpi/vendor/laminas/laminas-mail/src/Protocol)
laminas/laminas-mime in (/var/www/clients/client1/web1/web/glpi/vendor/laminas/laminas-mime/src)
rlanvin/php-rrule in (/var/www/clients/client1/web1/web/glpi/vendor/rlanvin/php-rrule/src)
ramsey/uuid in (/var/www/clients/client1/web1/web/glpi/vendor/ramsey/uuid/src)
psr/log in (/var/www/clients/client1/web1/web/glpi/vendor/psr/log/Psr/Log)
psr/simple-cache in (/var/www/clients/client1/web1/web/glpi/vendor/psr/simple-cache/src)
psr/cache in (/var/www/clients/client1/web1/web/glpi/vendor/psr/cache/src)
league/csv in (/var/www/clients/client1/web1/web/glpi/vendor/league/csv/src)
mexitek/phpcolors in (/var/www/clients/client1/web1/web/glpi/vendor/mexitek/phpcolors/src/Mexitek/PHPColors)
guzzlehttp/guzzle in (/var/www/clients/client1/web1/web/glpi/vendor/guzzlehttp/guzzle/src)
guzzlehttp/psr7 in (/var/www/clients/client1/web1/web/glpi/vendor/guzzlehttp/psr7/src)
glpi-project/inventory_format in (/var/www/clients/client1/web1/web/glpi/vendor/glpi-project/inventory_format/lib/php)
wapmorgan/unified-archive in (/var/www/clients/client1/web1/web/glpi/vendor/wapmorgan/unified-archive/src)
paragonie/sodium_compat in (/var/www/clients/client1/web1/web/glpi/vendor/paragonie/sodium_compat/src)
symfony/cache in (/var/www/clients/client1/web1/web/glpi/vendor/symfony/cache)
html2text/html2text in (/var/www/clients/client1/web1/web/glpi/vendor/html2text/html2text/src)
symfony/css-selector in (/var/www/clients/client1/web1/web/glpi/vendor/symfony/css-selector)
symfony/dom-crawler in (/var/www/clients/client1/web1/web/glpi/vendor/symfony/dom-crawler)
twig/twig in (/var/www/clients/client1/web1/web/glpi/vendor/twig/twig/src)
twig/string-extra in (/var/www/clients/client1/web1/web/glpi/vendor/twig/string-extra)
symfony/polyfill-ctype not found
symfony/polyfill-iconv not found
symfony/polyfill-mbstring not found
symfony/polyfill-php80 not found
symfony/polyfill-php81 not found
symfony/polyfill-php82 in (/var/www/clients/client1/web1/web/glpi/vendor/symfony/polyfill-php82)
league/oauth2-client in (/var/www/clients/client1/web1/web/glpi/vendor/league/oauth2-client/src/Provider)
league/oauth2-google in (/var/www/clients/client1/web1/web/glpi/vendor/league/oauth2-google/src/Provider)
thenetworg/oauth2-azure in (/var/www/clients/client1/web1/web/glpi/vendor/thenetworg/oauth2-azure/src/Provider)
phpCas version 1.3.8 in (/usr/share/php/CAS/source)

LDAP directories
 
deleted for data protection reasons

SQL replicas
 
Not active

Notifications
 
Way of sending emails: PHP

Plugins list
 

Anything else?

No response

cedric-anne commented 4 months ago

Hi,

Could you please provide an email, into the .eml format, that can be used to reproduce the issue ? We need the email generated in your step 2 (the email that contains a .msg file as attachement).

schieffahrer commented 4 months ago

How could I send the data without running into data protection problems? There is some personal data in the text.

I was able to find out something else in the meantime: After the MSG file is sent to the ticket system, the ticket system replies with the ticket number and the content. The attachment is also included here. This is then called "msg_1_wg-your-scim-token-will-expire-soon-testmail.EML.msg" and the file type is a MSG file and not an EML file. It is the same file that I sent. When I open the ticket system, the file is called "msg_1_wg-your-scim-token-will-expire-soon-testmail.EML" and has been converted to EML. The sender therefore only sees their own file but not the file in the ticket system.

cedric-anne commented 4 months ago

How could I send the data without running into data protection problems?

It is not possible in the community support context. Either you have to ask for professionnal support, either you have to reproduce the issue using an email account created for this purpose only.

plbv commented 4 months ago

I am experiencing almost the same issue with HTML code in email, using GLPI 10.0.14 and the IMAP collector.

I have tested various scenarios. On my end, it appears that the Outlook application (both Outlook 365 desktop and Outlook 2019) is responsible for this .MSG to .EML conversion. The new ticket email notification from GLPI has the attachment file already in .EML format. Some of the testing:

Attached are three files for reference:

  1. One MSG file to reproduce the issue. When attached to an email from the Outlook desktop app to GLPI, downloaded from GLPI to local, and opened in Outlook, the HTML will be partially broken.
  2. Two EML files: one downloaded from GLPI (HTML partially broken) and one from mailbox directly from OWA (HTML displays correctly).

glpi.zip

Some differences in the header: .MSG without issue:

Content-Type: multipart/alternative;
    boundary="----=_NextPart_000_001C_01DAD47C.93C86CF0"

.EML without issue:

Content-Type: multipart/alternative;
    boundary="_000_001b01dad49e1ad949a0508bdce0gmailcom_"

.EML with issue:

Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Would it have anything to do with the windows-1252 charset, if that's what Outlook is now using?

schieffahrer commented 4 months ago

I have found the problem with the display in Outlook. When converting to a MSG file, Unix line endings are typically used “\n”. Windows expects “\r\n” and therefore displays the file incorrectly. If I take the same EML file and convert the line endings, it is also displayed correctly in Outlook. msg_1_testmail.zip

The screenshot on the left shows the file from the GLPI, on the right the line endings have been changed on the same data. The left-hand display is incorrect and the embedded image is displayed as an attachment. The characters you see in the text are also partially incorrect. 2024-07-23 09_32_04-i am a test Subject

schieffahrer commented 4 months ago

By the way, the error is exclusively in Microsoft Outlook. The problem does not exist in Thunderbird or other EML viewers and the display is correct.

github-actions[bot] commented 2 months ago

There has been no activity on this issue for some time and therefore it is considered stale and will be closed automatically in 10 days.

If this issue is related to a bug, please try to reproduce on latest release. If the problem persist, feel free to add a comment to revive this issue. If it is related to a new feature, please open a topic to discuss with community about this enhancement on suggestion website.

You may also consider taking a subscription to get professionnal support or contact GLPI editor team directly.