Cannot open files with special chars on filename

Mer0me commented 1 week ago

How to use GitHub

Please use the 👍 reaction to show that you are affected by the same issue.
Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
Subscribe to receive notifications on status change and new comments.

Steps to reproduce

Upload an Outlook .msg file to Nextcloud
Try to open it
Blank page and errors in console

Expected behaviour

The app should open .MSG as it does for .EML files.

Actual behaviour

Only .EML files are opened.

Server configuration

Web server: Apache

Database: MariaDB

PHP version: 8.3

Nextcloud version: 30.0.2

List of activated apps

``` Enabled: - activity: 3.0.0 - admin_audit: 1.20.0 - calendar: 5.0.1 - calendar_resource_management: 0.8.0 - circles: 30.0.0 - cloud_federation_api: 1.13.0 - comments: 1.20.1 - contacts: 6.1.0 - contactsinteraction: 1.11.0 - dashboard: 7.10.0 - dav: 1.31.1 - deck: 1.14.1 - external: 5.5.2 - federatedfilesharing: 1.20.0 - federation: 1.20.0 - files: 2.2.0 - files_downloadlimit: 3.0.0 - files_emailviewer: 0.1.2 - files_external: 1.22.0 - files_linkeditor: 1.1.21 - files_mindmap: 0.0.30 - files_pdfviewer: 3.0.0 - files_reminders: 1.3.0 - files_sharing: 1.22.0 - files_texteditor: 2.15.1 - files_trashbin: 1.20.1 - files_versions: 1.23.0 - forms: 4.3.2 - guests: 4.0.1 - impersonate: 1.17.0 - integration_giphy: 2.0.1 - integration_mastodon: 2.0.6 - integration_openstreetmap: 2.0.1 - logreader: 3.0.0 - lookup_server_connector: 1.18.0 - mail: 4.0.2 - news: 24.0.0 - nextcloud_announcements: 2.0.0 - notifications: 3.0.0 - notify_push: 0.7.0 - oauth2: 1.18.1 - onlyoffice: 9.5.0 - password_policy: 2.0.0 - passwords: 2024.11.20 - phonetrack: 0.8.1 - photos: 3.0.2 - polls: 7.2.4 - privacy: 2.0.0 - provisioning_api: 1.20.0 - related_resources: 1.5.0 - serverinfo: 2.0.0 - settings: 1.13.0 - sharebymail: 1.20.0 - spreed: 20.0.2 - support: 2.0.0 - systemtags: 1.20.0 - tasks: 0.16.1 - text: 4.1.0 - theming: 2.5.0 - twofactor_backupcodes: 1.19.0 - updatenotification: 1.20.0 - user_status: 1.10.0 - viewer: 3.0.0 - weather_status: 1.10.0 - webhook_listeners: 1.1.0-dev - workflowengine: 2.12.0 ```

Nextcloud configuration

``` { "system": { "enable_previews": true, "enabledPreviewProviders": [ "OC\\Preview\\Movie", "OC\\Preview\\PNG", "OC\\Preview\\JPEG", "OC\\Preview\\GIF", "OC\\Preview\\BMP", "OC\\Preview\\XBitmap", "OC\\Preview\\MP3", "OC\\Preview\\MP4", "OC\\Preview\\TXT", "OC\\Preview\\MarkDown", "OC\\Preview\\PDF" ], "instanceid": "***REMOVED SENSITIVE VALUE***", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "mycloud.net", "cloud.mycloud.net", "192.168.1.100" ], "datadirectory": "***REMOVED SENSITIVE VALUE***", "overwrite.cli.url": "https:\/\/cloud.mycloud.net", "dbtype": "mysql", "version": "30.0.2.2", "logtimezone": "UTC", "installed": true, "theme": "", "loglevel": 3, "maintenance": false, "trashbin_retention_obligation": "auto", "dbname": "***REMOVED SENSITIVE VALUE***", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbuser": "***REMOVED SENSITIVE VALUE***", "dbpassword": "***REMOVED SENSITIVE VALUE***", "mail_from_address": "***REMOVED SENSITIVE VALUE***", "mail_smtpmode": "sendmail", "mail_domain": "***REMOVED SENSITIVE VALUE***", "updater.release.channel": "stable", "mail_sendmailmode": "smtp", "app_install_overwrite": [ "calendar", "onlyoffice", "spreed", "files_mindmap", "files_texteditor", "external", "deck", "passwords", "emlviewer", "news", "groupfolders", "openotp_sign", "yumisign_nextcloud", "files_linkeditor", "files_emlviewer", "extract", "files_rightclick", "forms", "files_zip" ], "mysql.utf8mb4": true, "memcache.local": "\\OC\\Memcache\\APCu", "has_rebuilt_cache": true, "app.mail.verify-tls-peer": false, "default_phone_region": "FR", "debut": true, "memcache.locking": "\\OC\\Memcache\\Redis", "filelocking.enabled": "true", "redis": { "host": "***REMOVED SENSITIVE VALUE***", "port": 0, "timeout": 0 }, "trusted_proxies": "***REMOVED SENSITIVE VALUE***", "defaultapp": "files", "maintenance_window_start": 1 } } ```

Browser

Browser name: Firefox

Browser version: 132.0.1

Operating system: Ubuntu

Browser log

``` XHRGET https://cloud.mycloud.net/index.php/apps/files_emailviewer/convert?fileId=1119723 [HTTP/1.1 500 Internal Server Error 237ms] Réponse inattendue du serveur. PDF.js v3.11.174 (build: ce8716743) Message: Unexpected server response (500) while retrieving PDF "https://cloud.mycloud.net/index.php/apps/files_emailviewer/convert?fileId=1119723". [app.js:1249:12](webpack://pdf.js/web/app.js) Uncaught (in promise) Object { message: 'Unexpected server response (500) while retrieving PDF "https://cloud.mycloud.net/index.php/apps/files_emailviewer/convert?fileId=1119723".', name: "UnexpectedResponseException", status: 500, stack: "BaseExceptionClosure@https://cloud.mycloud.net/apps/files_pdfviewer/js/pdfjs/build/pdf.js?v=3.0.0:446:29\n__webpack_modules__<@https://cloud.mycloud.net/apps/files_pdfviewer/js/pdfjs/build/pdf.js?v=3.0.0:449:2\n__w_pdfjs_require__@https://cloud.mycloud.net/apps/files_pdfviewer/js/pdfjs/build/pdf.js?v=3.0.0:17849:41\n@https://cloud.mycloud.net/apps/files_pdfviewer/js/pdfjs/build/pdf.js?v=3.0.0:18129:32\n@https://cloud.mycloud.net/apps/files_pdfviewer/js/pdfjs/build/pdf.js?v=3.0.0:18140:3\n@https://cloud.mycloud.net/apps/files_pdfviewer/js/pdfjs/build/pdf.js?v=3.0.0:18143:12\nwebpackUniversalModuleDefinition@https://cloud.mycloud.net/apps/files_pdfviewer/js/pdfjs/build/pdf.js?v=3.0.0:31:50\n@https://cloud.mycloud.net/apps/files_pdfviewer/js/pdfjs/build/pdf.js?v=3.0.0:32:3\n" } ```

kesselb commented 1 week ago

Thank you for taking the time to try files_emailviewer.

Does it work to convert the msg file with https://github.com/nickrussler/email-to-pdf-converter?

kesselb commented 1 week ago

Also is there anything in your nextcloud.log?

Mer0me commented 1 week ago

Yes, the conversion is well done from the command line, I get the PDF corresponding to the MSG. Tested with different MSG files, it's not one particular .MSG that causes the problem.

Mime Structure of /var/www/nextcloud/data/USER/files/Documents/test.msg:
-----------Mime Message-----------
> multipart/related
> |  multipart/alternative
> |  |  text/plain
> |  |  text/html
> |  image/png; inline
----------------------------------
Extract the inline images
Start conversion to pdf
The switch --viewport-size, is not support using unpatched qt, and will be ignored.The switch --image-quality, is not support using unpatched qt, and will be ignored.QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
Loading page (1/2)
Printing pages (2/2)                                               
Done                                                           
Conversion finished

Nothing in nextcloud.log

kesselb commented 1 week ago

Please change the loglevel in config.php to 2 (warning) to log the error.

https://github.com/nextcloud/files_emailviewer/blob/719ff0889cb7d1ed2124bf227bb735d353d1c6b2/lib/Controller/PageController.php#L68-L73

Mer0me commented 1 week ago

OK, now I can see the warning "Conversion did not finish" :

{"reqId":"U2dCm46fZhszAi1vBZft","level":2,"time":"2024-11-11T13:25:55+00:00","remoteAddr":"192.168.1.99","user":"USER","app":"files_emailviewer","method":"GET","url":"/index.php/apps/files_emailviewer/convert?fileId=1119722","message":"Conversion did not finish","userAgent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:132.0) Gecko/20100101 Firefox/132.0","version":"30.0.2.2","exception":{"Exception":"OCA\\Files_EmailViewer\\Exception\\ConversionException","Message":"Conversion did not finish","Code":0,"Trace":[{"file":"/var/www/nextcloud/apps/files_emailviewer/lib/Controller/PageController.php","line":69,"function":"convert","class":"OCA\\Files_EmailViewer\\Service\\ConversionService","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":208,"function":"convert","class":"OCA\\Files_EmailViewer\\Controller\\PageController","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":114,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/App.php","line":161,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/Route/Router.php","line":302,"function":"main","class":"OC\\AppFramework\\App","type":"::"},{"file":"/var/www/nextcloud/lib/base.php","line":1001,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/var/www/nextcloud/index.php","line":24,"function":"handleRequest","class":"OC","type":"::"}],"File":"/var/www/nextcloud/apps/files_emailviewer/lib/Service/ConversionService.php","Line":58,"message":"Conversion did not finish","exception":{},"CustomMessage":"Conversion did not finish"}}

kesselb commented 1 week ago

https://github.com/nextcloud/files_emailviewer/blob/719ff0889cb7d1ed2124bf227bb735d353d1c6b2/lib/Service/ConversionService.php#L57-L59

The process output did not contain "Conversion finished" ;)

You can change the code to also log the stdout like below:

        if (str_contains($stdout, 'Conversion finished') === false) {
            throw new ConversionException('Conversion did not finish. Output: ' . $stdout);
        }

Mer0me commented 1 week ago

Ok, it seems to be relative to the filename encoding.

$stdout is empty when it fails, but if I rename the file; skipping all special chars in filename, the same msg file is actually opened.

Opening "test.msg" => It works Renaming "test.msg" to "testé.msg" and then opening "testé.msg" => $stdout is empty, $filepath contains "/var/www/nextcloud/data/USER/files/Documents/test\u00e9.msg"

So, when the filename (maybe the whole path ?) contains some special char, I guess it cannot find the file and execute the conversion.

XueSheng-GIT commented 1 week ago

Duplicate of https://github.com/nextcloud/files_emailviewer/issues/29?

kesselb commented 1 week ago

@Mer0me @XueSheng-GIT mind to test the patch below?

Index: lib/Service/ConversionService.php
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/lib/Service/ConversionService.php b/lib/Service/ConversionService.php
--- a/lib/Service/ConversionService.php (revision 719ff0889cb7d1ed2124bf227bb735d353d1c6b2)
+++ b/lib/Service/ConversionService.php (date 1731422667211)
@@ -42,7 +42,12 @@
            $resultPath
        ];

-       $process = proc_open($command, $descriptors, $pipes);
+       $env = [
+           'LANG=C.UTF-8',
+           'LC_ALL=C.UTF-8',
+       ];
+
+       $process = proc_open($command, $descriptors, $pipes, null, $env);
        if ($process === false) {
            throw new ConversionException('Could not invoke emailconverter.jar');
        }

Mer0me commented 1 week ago

It works for me ! Thanks a lot !

nextcloud / files_emailviewer