Allow more preview formats

[rullzer]

Right now we serve previews in png or jpg. Which has the advantage of being support on nearly every platform since the late 90s. But of course it is no longer the best solution out there.

I would propose to allow different image formats. The top candidate in my opinion is webp. This allows for smaller images at comparable quality. Of course the backend would need to distinquis what to server but I'm sure we can solve that with some js magic to set the accept headers ;)

This doesn't mean we fully get rid of the jpg an png as older firefox and IE versions do not support it. But I bet this could be performance boost especially when using a lot of images.

[J0WI]

There is also GraphicsMagick as an alternative to ImageMagick. Vips would be a very fast option but it's not yet implemented in Nextcloud (stale effort in #28279).

10-bit support depends on whether the library was compiled to support it (e.g. CONFIG_AV1_HIGHBITDEPTH=1 in aom).

[JanisPlayer]

@J0WI I think it's possible to replace ImageMagick with GraphicsMagick. It's easily usable in PHP with almost the same functionality. Vips also sounds interesting, but it's more complex to use. However, the current functionality of the Preview Generator won't automatically switch all formats to GraphicsMagick. This means you'd not only have to rewrite existing ImageMagick Preview Generators as a fallback for GD but also add new ones. Whether the performance boost is worth it is the next question. Since a new solution for fast generation seems necessary soon, especially considering Imaginary is no longer being developed by the original developer. It might be used for modernization over time. Just a speculation. Thanks for the acceleration tips; I can use those elsewhere.

[DesertCookie]

@JanisPlayer I'm facing an issue with your patch. Running the pre-generate I get PHP Fatal error: Paletter image not supported by webp in /var/www/html/lib/private/legacy/OC_Image.php on line 455. Running with -vvv indicates it's failing for a document preview: 2024-02-24T17:07:27+00:00 Generating previews for /34/files/Vorlagen/Expense report.ods (only other log output before the error).

I am unsure whether it is an issue with your patch at all, or possibly my instance/install: Fresh instance of Nextcloud AIO (less than a week old) with v27.1.7 RC1 (your patch is only for v27.1.2). Preview format is set to WebP with a quality of 30. At least since applying your patch I don't even get previews downloaded (or possibly even generated) to my Desktops via the desktop client for images (usually it does load a preview, even when the file is only present online).

[JanisPlayer]

I took a brief look at the file. At this point, it checks if your PHP version supports GD WebP with imagetypes() & IMG_WEBP. If not, it returns an error, and the generation is aborted. For Nextcloud AIO, I haven't tested the patch yet due to some security-related removal of certain preview functions like those involving ImageMagick. However, a quick fix for this issue is using Imaginary.php for generation, which I've included in older patches, modified for WebP generations. This change is present in the latest Nextcloud version. I using the old patch for version 27.1 multiple times, even for newer versions where not much has changed, could potentially lead to errors. I suspect the issue is related to the PHP version and installed modules. WebP and AVIF are supported under GD only in later PHP versions. It's strange that a new Nextcloud version doesn't use the recommended PHP version. I'll create a simple script soon to automatically add the changes during pattern search. Handling this for apps is a bit more complex since the code needs processing, but it can work with a pattern patch. I'll check later which PHP version is used in the AIO version.

Edit: https://github.com/nextcloud/all-in-one So I looked, the latest Docker version of them uses the libs, a PHP version that should be able to do that and also the modules.

[DesertCookie]

Trying again today, I do not get any log output after running occ preview:pre-generate -vvv. I've tried with the preview format set to webp, avif, and removing it. I also don't get AVIF previews in the web interface.

[JanisPlayer]

The errors may have disappeared because images had already been generated. But since the patch is actually intended more for manual installation, this may also be due to differences compared to all in one installation.Since there is no way to install it, can you give me more information about your installation and how you ran the patch? So it can actually only be due to a missing lib, module or incorrect PHP version. Or that the version is actually incompatible with the patch. But since WebP previews are enough for you, in version 28 works without a patch. You can read about how Imaginary does this in the documentation. But if you really want AVIF or want to create the previews via GD, this is currently not possible without a patch. Unfortunately, I don't have any information about when a review is planned.

[DesertCookie]

WebP would be enough for me. However, Nextcloud AIO doesn't allow me to upgrade to 28 yet.

I simply installed the nextcloud-aio-mastercontainer:nextcloud/all-in-one in unRAID 6.12.6 and allowed it to configure all the containers for me: nextcloud-aio-database:nextcloud/aio-postgresql, nextcloud-aio-redis:nextcloud/aio-redis, nextcloud-aio-nextcloud:nextcloud/aio-nextcloud, nextcloud-aio-apache:nextcloud/aio-apache, nextcloud-aio-docker-socket-proxy:nextcloud/nextcloud-aio-docker-socket-proxy, nextcloud-aio-clamav:nextcloud/aio-clamav, nextcloud-aio-fulltextsearch:nextcloud:aio-fulltextsearch, nextcloud-aio-imaginary:nextcloud/aio-imaginary, nextcloud-aio-talk-recording:nextcloud/atio-talk-recording, nextcloud-aio-talk:nextcloud/aio-talk, nextcloud-aio-notify-push:nextcloud/aio-notify-push, nextcloud-aio-watchtower:nextcloud/aio-watchtower. I also run an OnlyOffice server.
Everything is proxied by Nginx Proxy Manager. Logins are handled via Authentik SAML. Everything is stored in a Docker volume on my unRAID server's cache SSD. The nextcloud-aio-nextcloud runs PHP 8.1.27.

I simply downloaded the v27 patch ZIP and replaced the files in my volume with the ones from the patch. Then I restarted the container (Nextcloud and Apache).

[DesertCookie]

@JanisPlayer the webp_quality does not change the quality of the generated preview files for me. I have tried anything from 30 to 60 and keep getting webp preview files with the same quality (within 2%). I set the option both in the config file and via your provided OCC command.

[JanisPlayer]

At this point, the default value is defined, which is used if no value is set via occ config:app:set preview avif_quality --value="30". https://github.com/JanisPlayer/server/blob/b5fa45a083e7ff302189343965c33015998b7f0d/lib/private/legacy/OC_Image.php#L58 This line checks if a value is set. https://github.com/JanisPlayer/server/blob/b5fa45a083e7ff302189343965c33015998b7f0d/lib/private/legacy/OC_Image.php#L505 After that, your image is generated with the desired quality: https://github.com/JanisPlayer/server/blob/b5fa45a083e7ff302189343965c33015998b7f0d/lib/private/legacy/OC_Image.php#L460 So, at this point, without logs and precise debugging, I can only speculate why it doesn't work for you to change the quality. It might be due to a compatible version with your Nextcloud version. Since I'm not sure if the patch will be accepted anyway. The only information I found is that AVIF support is planned from version 30 onwards. https://github.com/nextcloud/server/pull/37564#issuecomment-2006968272 Unfortunately, the developer no longer feels like working on his PR, although I thought he would add what I built here as just a feasibility test, making it nicer and more versatile for other formats. So I'll probably have to improve this feasibility test later, but so far, I have no information on what will be added and desired. That's why I haven't improved the quickly improvised feasibility test yet, which only makes minimal changes to the preview generator. So, what my PR should actually do is just ensure that AVIF images are also displayed in the photo app, processed correctly in the AI apps, and previews are created from AVIF images. So basically, the feasibility test should also work. I have also considered just writing an app for this or temporarily using a script that simply replaces the code according to a pattern and executes NodeJS for the apps. So I can only suggest applying the changes to your Nextcloud by doing it yourself if I haven't made a patch for your version yet. You can see the changes here: https://github.com/nextcloud/server/pull/40048/files, you can also use a code comparison tool to compare the differences between the patch and your code and incorporate them into your code. So maybe this information will help you solve the problem.

[Forza-tng]

Hi. I just want to double check if this patch series enables using avif or webp as format for the previews, even if the original image is a jpeg? I have just upgraded to NC 29 and thought it would have a go at regenerating my previews.

I have not applied your patches yet. This is current status for my system after removing all previews and regenerating them using occ preview:generate-all. I am not using Imaginary at the moment. I have PHP 8.2.20 and ImageMagick 7.1.1.

# count of preview files 
❯ find -type f -name "*.webp" | wc -l
301
❯ find -type f -name "*.jpg" | wc -l
298441

config.php

'enable_previews' => true,
'preview_format' => 'webp',

[JanisPlayer]

WebP for previews is already supported in the current version of Nextcloud via Imaginary. AVIF support is still in development but already functional; however, the last tested patch I created was for Nextcloud (28.0.1). You can create a new patch by simply applying all changes to the current version. I wanted to automate this process, but for instance, the Viewer app needs to be rebuilt with NodeJS each time to display AVIF images, which became too time-consuming for me. Another issue is that developers haven't communicated to me what else is needed. For example, there was also the idea of incorporating @aentwist's PR into mine to provide more flexibility in selection, or whether to retain a somewhat imperfect PoW concept or focus solely on displaying AVIF images without AVIF or WebP previews. Therefore, my PR is not currently being actively maintained. However, it has generally been kept up-to-date enough to be easily added to the current version. So, if you want to create a patch for version 29, you simply need to use Diff to add the changes from my patch to the files. The Viewer app needs to be rebuilt with my changes to display AVIF images. For the Photos app, it suffices to modify the file without needing to rebuild the app. Don't forget to add AVIF images to the list as well. Nextcloud config example:

'enable_previews' => true,
'preview_format' => 'webp',
# For Imaginary: 'preview_imaginary_url' => 'http://127.0.0.1:9000',
'enabledPreviewProviders' => 
array (
# For Imaginary: 0 => 'OC\Preview\Imaginary',
1 => 'OC\Preview\JPEG',
2 => 'OC\Preview\GIF',
3 => 'OC\Preview\BMP',
4 => 'OC\Preview\XBitmap',
5 => 'OC\Preview\Movie',
6 => 'OC\Preview\PDF',
7 => 'OC\Preview\MP3',
8 => 'OC\Preview\TXT',
9 => 'OC\Preview\MarkDown',
10 => 'OC\Preview\PNG',
11 => 'OC\Preview\WebP',
12 => 'OC\Preview\HEIC',
13 => 'OC\Preview\OpenDocument',
14 => 'OC\Preview\Krita',
15 => 'OC\Preview\AVIF',
20 => 'OC\Preview\Movie',
),

If anyone wants to automate this process, for example, using Bash, they can share the script with me. But I'm not sure if there is enough demand to justify it. Ideally, this PR should have been included in Nextcloud long ago.

[Forza-tng]

WebP for previews is already supported in the current version of Nextcloud via Imaginary.

I see..I might look at Imaginary again. I just didn't see a huge benefit, and it also had problems with some files that otherwise worked. I think, for example, dng/raw files did not work with Imaginary.

I am surprised that Imaginary is a requirement for WebP as PHP natively supports it in its gd library since a while back. I created a ticket for allowing WebP previews via PHP gd. In PHP 8.1 even AVIF is natively supported. https://github.com/nextcloud/server/issues/46611

[jmigual]

Hey! So I'm facing a strange issue. Somehow, even though I enabled webp preview creation, I still see some previews generated as .jpg. I'ved made sure to check whether they were from before by deleting all .jpg images but they appear again after a while (when nextcloud generates them). This is the relevant part of my config:

config.php

```php false, 'default_language' => 'ca', 'default_locale' => 'ca_ES', 'default_phone_region' => 'ES', 'defaultapp' => 'files', 'htaccess.RewriteBase' => '/', 'memcache.local' => '\\OC\\Memcache\\APCu', 'allow_local_remote_servers' => true, 'apps_paths' => array ( 0 => array ( 'path' => '/var/www/html/apps', 'url' => '/apps', 'writable' => false, ), 1 => array ( 'path' => '/var/www/html/custom_apps', 'url' => '/custom_apps', 'writable' => true, ), ), 'memcache.distributed' => '\\OC\\Memcache\\Redis', 'memcache.locking' => '\\OC\\Memcache\\Redis', 'passwordsalt' => 'REDACTED', 'secret' => 'REDACTED', 'trusted_domains' => array ( 0 => 'localhost', 1 => 'REDACTED', 2 => 'REDACTED', ), 'datadirectory' => '/var/www/html/data', 'version' => '28.0.7.4', 'overwrite.cli.url' => 'REDACTED', 'dbtype' => 'pgsql', 'dbname' => 'nextcloud', 'dbhost' => 'nextcloud-postgres', 'dbport' => '', 'dbtableprefix' => 'oc_', 'dbuser' => 'nextcloud', 'dbpassword' => 'REDACTED', 'mysql.utf8mb4' => true, 'installed' => true, 'instanceid' => 'REDACTED', 'maintenance' => false, 'overwriteprotocol' => 'https', 'loglevel' => 0, 'logfile' => '/var/www/html/data/logs/nextcloud.log', 'trashbin_retention_obligation' => '30, 31', 'versions_retention_obligation' => 'auto, 31', 'enabledPreviewProviders' => array ( 0 => 'OC\\Preview\\GIF', 1 => 'OC\\Preview\\Imaginary', 2 => 'OC\\Preview\\PNG', 3 => 'OC\\Preview\\WebP', 4 => 'OC\\Preview\\JPEG', 5 => 'OC\\Preview\\HEIC', 6 => 'OC\\Preview\\BMP', 7 => 'OC\\Preview\\XBitmap', 8 => 'OC\\Preview\\MP3', 9 => 'OC\\Preview\\TXT', 10 => 'OC\\Preview\\MarkDown', 11 => 'OC\\Preview\\Krita', 12 => 'OC\\Preview\\MP4', 13 => 'OC\\Preview\\Movie', 14 => 'OC\\Preview\\MKV', 15 => 'OC\\Preview\\AVI', 16 => 'OC\\Preview\\PDF', 17 => 'OC\\Preview\\SVG', 18 => 'OC\\Preview\\TIFF', ), 'enable_previews' => true, 'preview_max_x' => 4096, 'preview_max_y' => 4096, 'preview_imaginary_url' => 'http://imaginary:9000', 'preview_format' => 'webp', 'preview_ffmpeg_path' => '/usr/bin/ffmpeg', 'webp_quality' => '80', 'jpeg_quality' => 80, 'app_install_overwrite' => array ( 0 => 'facerecognition', 1 => 'extract', ), 'theme' => '', 'mail_smtpmode' => 'smtp', 'mail_smtphost' => 'REDACTED', 'mail_sendmailmode' => 'smtp', 'mail_from_address' => 'nextcloud', 'mail_domain' => 'REDACTED', 'mail_smtpauth' => 1, 'mail_smtpauthtype' => 'LOGIN', 'mail_smtpname' => 'apikey', 'mail_smtppassword' => 'REDACTED', 'mail_smtpport' => '587', 'memories.exiftool' => '/var/www/html/custom_apps/memories/bin-ext/exiftool-amd64-glibc', 'memories.vod.path' => '/var/www/html/custom_apps/memories/bin-ext/go-vod-amd64', 'memories.vod.ffmpeg' => '/usr/bin/ffmpeg', 'memories.vod.ffprobe' => '/usr/bin/ffprobe', 'redis' => array ( 'host' => 'nextcloud-redis', 'password' => '', 'port' => 6379, ), 'memories.gis_type' => 2, 'memories.db.triggers.fcu' => true, 'maintenance_window_start' => 1, 'trusted_proxies' => array ( 0 => 'REDACTED', ), ); ```

[JanisPlayer]

@jmigual There should be an error in the log when dealing with image formats that Imaginary does not support. As a fallback, the GD or Imagemagick generator is used, both of which do not support WebP. It might also be that your Imaginary has too low a limit on the number of images that can be generated simultaneously. This will also appear as an error message in the log. You can increase the limit in the Docker config. With your log and information about the image properties of the files that do not work and the config of your Imaginary container, we might be able to get to the bottom of the problem.

@Forza-tng Yes, that's exactly what my pull request does. The code of Preview Generator in its original version is very limited and only allows a few formats, so it's not PHP's fault. A modification for Imaginary was easier back then than rewriting the entire GD generator. I am currently using WebP with 80 percent quality via Imaginary myself.