search

nextcloud / gallery

DEPRECATED Gallery app was replaced by Photos
https://github.com/nextcloud/photos
GNU Affero General Public License v3.0
108 stars 58 forks source link

Unsupported JPEG stops the Gallery App from showing pictures #210

Open dtrumpfheller opened 7 years ago

dtrumpfheller commented 7 years ago

Bug report

Unsupported JPEG stops the Gallery App in the grid view from showing pictures after the unsupported one.

Steps to reproduce

  1. Add multiple JPEG pictures
  2. Add unsupported JPEG picture. In my case the unsupported picture is a panorama took with a Samsung Galaxy S7
  3. Add multiple JPEG pictures

Expected behaviour

Show all pictures. If some pictures are not supported then just skip them.

Actual behaviour

Only pictures till the unsupported one are shown. The spinner which indicates that the unsupported JPEG is being processes stays forever.

Server configuration

Operating system: Debian Jessie

Web server: nginx

Database: MariaDB

PHP version: 5.7 (from Debian Stretch)

cloud server: Nextcloud or ownCloud Nextcloud

cloud version: (see admin page or version.php) 11.0.1

Updated from an older installation or fresh install: Updated

List of activated apps: Gallery App, ...

The content of config/config.php:

Are you using external storage, if yes which one: local folder, smb share, sftp, etc. no

Are you using encryption: yes/no no

Are you using custom gallery.cnf config files: yes/no no

Web server error log

none

Cloud log

imagecreatefromjpeg(): gd-jpeg: JPEG library reports unrecoverable error: Unsupported JPEG process: SOF type 0xc8 at /var/www/nextcloud/lib/private/legacy/image.php#538 imagecreatefromjpeg(): gd-jpeg: JPEG library reports unrecoverable error: Unsupported marker type 0xf3 at /var/www/nextcloud/lib/private/legacy/image.php#538

Client configuration

Browser: Firefox

Operating system: Windows 7

Browser log

None

oparoz commented 7 years ago

This is an issue with the code in server, but I don't think anybody can do anything about it. I believe it has been fixed in the GD library itself, so keep updating your server and close the issue once this has been resolved for you.

dtrumpfheller commented 7 years ago

Hello,

there are two problems. Problem 1) The generation of the preview fails. It sounds like it is done in a different lib, so it can't be solved here. So fine so good.

Problem 2) An exception during preview generation leads to a not-working gallery app. In my eyes this is the result of a missing or wrong exception handling within the gallery app. The app should catch the exception and proceed with the next image.

Greetings

oparoz commented 7 years ago

The app should catch the exception and proceed with the next image.

That's what we do, but when PHP generates a fatal error, everything stops. I haven't found a solution for that problem.

linuxrrze commented 7 years ago

I did some research on this topic, and I found, that the wrong error handling seems to occur on a lower level:

When trying to load / convert images that cause the above error with other tools like "gimp" or "jpegtran" I get very similar errors.

So I guess the error is located somewhere deep in a basic (jpeg?) library ...

In my case the error is often caused by panorama photos taken by my smartphone.

My environment: Running Ubuntu 16.04 (x86_64), Nextcloud 11.0.2

godfuture commented 7 years ago

This issue happens for me as well.

I try to generate all thumbnails sudo -u www-data php /var/www/nextcloud/occ preview:generate-all

and the job always fails at the same folder (or lets say at the same photos) being processed.

gd-jpeg: JPEG library reports unrecoverable error: Unsupported marker type 0x8c in /var/www/nextcloud/lib/private/legacy/image.php on line 538

Are there any news?

linuxrrze commented 7 years ago

Hi I just did a little more research on this topic, and to me it looks like the following commit to the libjpeg-turbo library from Jun 28, 2017 could point to the root of this problem:

https://github.com/libjpeg-turbo/libjpeg-turbo/commit/b0971e47d76fdb81270e93bbf11ff5558073350d

Unfortunately it also mentions that the problem could be hard to fix without API changes.

Note: I'm using Ubuntu 16.04 and the php7.0-gd package is using libjpeg-turbo8 as JPEG library:

# dpkg -L php7.0-gd
...
/usr/lib/php/20151012
/usr/lib/php/20151012/gd.so
...

# ldd  /usr/lib/php/20151012/gd.so | grep jpeg
    libjpeg.so.8 => /usr/lib/x86_64-linux-gnu/libjpeg.so.8 (0x00007f59d43ce000)
linuxrrze commented 7 years ago

It was mentioned above that this bug was fixed in the gd library, however I could only find 2 issues and a proposed fix (not included in latest stable 2.2.4):

https://github.com/libgd/libgd/issues/380 https://github.com/libgd/libgd/issues/338

Fix (proposed):

https://github.com/libgd/libgd/pull/346

I just build 2.2.4 Ubuntu packages with this fix included. Currently running

./occ preview:pre-generate

I'll report here about the results once it's finished.

linuxrrze commented 7 years ago

The above fix to libgd does not solve this problem:

Replacing libgd with version 2.2.4 (and fix) on Ubuntu 16.04 still fails on certain images (Galaxy S7 panorama).

If there's some other patch to try, please let me/us know.

linuxrrze commented 7 years ago

For panorama pictures created using a Galaxy S7 the file format can be fixed (in many cases) using this script:

https://gist.github.com/bcyrill/e59fda6c7ffe23c7c4b08a990804b269

Make sure to read the comments and apply the changes mentioned there (otherwise this will not work for new panorama fotos as the format changed during one of the recent android updates)

And make sure to have a backup of your files before trying!

pren0x commented 7 years ago

For panorama pictures created using a Galaxy S7 the file format can be fixed (in many cases) using this script: https://gist.github.com/bcyrill/e59fda6c7ffe23c7c4b08a990804b269 Make sure to read the comments and apply the changes mentioned there (otherwise this will not work for new panorama fotos as the format changed during one of the recent android updates) And make sure to have a backup of your files before trying!

Thanks! It works for me. It's not perfect, but I can use gallery app.

godfuture commented 7 years ago

In this thread #1732 they talk about other libraries like ImageMagick. As it is insecure, GraphicsMagick was proposed. Maybe GraphicsMagick has less of these issues?

Can someone try to use GraphicsMagick to generate a preview of the same problematic image? Or is this not really easy to test?

srkunze commented 7 years ago

Same problem here. What can we do here?

I incur the problem when issuing nextcloud.occ preview:generate-all: 'Unsupported marker type 0xfb' Unfortunately, it doesn't tell what image seems to be corrupted.

srkunze commented 7 years ago

Would it help to wrap the conversion into its own subprocess/thread?

BeehiveSystems commented 7 years ago

I am having this issue as well. It seems that it is indeed the Galaxy S7 Edge panorama photos causing the issue.

godfuture commented 7 years ago

I would like to create all previews with cmdline but can't, because my Samsung S4 Mini photos make the process crash: sudo -u www-data php /var/www/nextcloud/occ preview:generate-all

What is the status on this? The gallery app is for no use, if I have to wait for the previews all the time...

j-ed commented 6 years ago

The workaround given by @linuxrrze still works but it might be necessary to slightly modify the script so that it also supports newer versions of the panorama images as described here, This can be done by using the version string "105" instead of "103": https://gist.github.com/bcyrill/e59fda6c7ffe23c7c4b08a990804b269#gistcomment-1860291