openequella / openEQUELLA

Core openEQUELLA sources
https://openequella.github.io/
Apache License 2.0
42 stars 44 forks source link

Imagemagick chokes on attachments with UTF-8 characters in their names when creating thumbnails #247

Closed smclaughlin520 closed 5 years ago

smclaughlin520 commented 7 years ago

Maybe we can escape them, or it might also not be important at all.

COMMENTS IN ORIGINAL JIRA ProcessBuilder is supposed to be escaping arguments - this might get ugly if we need to start doing it in a cross-platform manner.

Could you attach the offending image? I tried a ' char in mine and it worked fine on linux, is the server windows?"

Try this = Using a windows server

Tested with r28492. Using http://testing.thelearningedge.com.au:8050/pg/happy/ I successfully contributed an item containing the following files: Peng'uins.jpg, 'Tulips'.jpg, Zero'ByteBug.png and Fix'e'd2.png. All behaved OK in FF and IE. BUT it came to grief when I tried the filename mentioned at the beginning of this bug. Here are the steps. 1) Take a standard jpg (such as the Koala.jpg in the Windows library) and rename it to the following: Tom O’Hern Yowie 2010 Indian ink on paper, acrylic box frame 95.5h x 70w cm sheet size, 101h x 71.5w cm.jpg 2) Create a Learning Resource item using the File Manager to attach the jpg you just renamed. 3) Save and Publish. (OK so far) 4) Click Search. The new item appears in the Search Results but WITHOUT a thumbnail. 5) Click the name of the new item, then click the long filename. This takes us to the LargeImageViewer with the following messages.

When we click Yes, the system churns for several minutes and eventually produces an image. 6) Return to the Search screen. The new item still has no thumbnail. Note: I tried again using a long filename but without the apostrophe. It worked OK. Moral: It seems to be the combination of an apostrophe AND a long filename that causes the bug."

And I just tried the following, all of which succeeded. ApC'o,m.mDot.jpg (apostrophe, comma, dot) Ap'S p.Dot.jpg (apostrophe, space, dot) ApS'p C,omm.jpg (apostrophe, space, comma) SpC o,m.Dot.jpg (space, comma, dot) ApSp'C o,m.Dot.jpg (apostrophe, space, comma, dot)

Here are the results of some more experiments. 1) Filenames having more than about 150 characters to the left of the "".jpg"" fail to produce thumbnails. (143 char is OK, 162 is not.) 2) In most cases this limit of about 150 char continues to apply when the filename contains apostrophes, commas, dots and spaces. e.g. The following produced a thumbnail OK. h143ujhytgzsxdcfv.;, sxdcfvgbhzsxdcfvgbhzsxdcfvgbhzsxdcfvgbhzsxdcfvgbhzsxdcfvgbhzsxdcfvgbhzsxdcfvgbhzsxdcfvgbhzsxdcfvgbhzsxdcfvgbhzsxdcfvgbhewq.jpg 3) BUT the original filename mentioned in this bug fails, even though it is only 98 char. Tom O'Hern Yowie 2010 Indian ink on paper, acrylic box frame 95h x 70w cm sheet size, 101h x 71w cm.jpg 4) And so does this. O'H.jpg"

I got a different error using the same filename:

09:53:32,118 ERROR [ThumbnailServiceImpl] Error generating thumbnail java.lang.RuntimeException: Exec process returned 1. StdOut:

StdErr: identify.exe: Empty input file `C:\filestores\trunkpostgres8\filestore\Staging\69\3933eb73-9869-f1e3-72b9-910e4843e633\Tom O’Hern Yowie 2010 Indian ink on paper, acrylic box frame 95.5h x 70w cm sheet size, 101h x 71.5w cm.jpg' @ error/jpeg.c/EmitMessage/235.

at com.tle.common.util.ExecUtils$ExecResult.ensureOk(ExecUtils.java:164) at com.tle.core.imagemagick.ImageMagickServiceImpl.getImageDimensions(ImageMagickServiceImpl.java:153) at com.tle.core.imagemagick.ImageMagickServiceImpl.generateThumbnailAdvanced(ImageMagickServiceImpl.java:39) at com.tle.core.imagemagick.ImageMagickServiceImpl.generateStandardThumbnail(ImageMagickServiceImpl.java:235) at com.tle.core.workflow.thumbnail.ImageThumbnailGenerator.generateThumbnail(ImageThumbnailGenerator.java:19) at com.tle.core.workflow.thumbnail.ThumbnailServiceImpl.makeThumbnail(ThumbnailServiceImpl.java:48) at com.tle.core.workflow.thumbnail.CreateThumbnailOperation.execute(CreateThumbnailOperation.java:64) at com.tle.core.services.item.impl.ItemServiceImpl.execute(ItemServiceImpl.java:858) at com.tle.core.services.item.impl.ItemServiceImpl.execute(ItemServiceImpl.java:800) at com.tle.core.services.item.impl.ItemServiceImpl.execute(ItemServiceImpl.java:806) at com.tle.core.services.item.impl.ItemServiceImpl.executeOperationsNow(ItemServiceImpl.java:1000) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:310)"


The actual command line as reported by the awesome ProcMon

""C:\Program Files\ImageMagick\identify.exe"" -format %wx%h ""C:\filestores\trunkpostgres8\filestore\Staging\60\7b754023-f5c3-992e-d2ff-7d95890cbcf8\Tom O’Hern Yowie 2010 Indian ink on paper, acrylic box frame 95.5h x 70w cm sheet size, 101h x 71.5w cm.jpg"""

It's time we had that file system where files are stored with no relation to their original filename...

I suspect it's this issue: http://studio.imagemagick.org/discourse-server/viewtopic.php?f=3&t=16767

Note that that ""quote"" in the filename is not your garden variety quote. It's some unicode thing that things like Word like to use.

CONFIRMED: changing the quote to an everyday straight-off-the-keyboard-quote works."

This also appears to be a bug in Java that is scheduled to be fixed in Java 7 (Isn't everything??) http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4947220 However, mucking about with codepages etc on the command line has not helped either, so it's probably also an issue in ImageMagick

The only solution to this is to encode all non-ASCII characters in filenames.

While you're at it, try uploading the attached file to the scrapbook. (Mond1 - â•«.png)


Still broken in 6.4 QA1

Issue Converted from original Equella JIRA

stale[bot] commented 5 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.