search

cantaloupe-project / cantaloupe

High-performance dynamic image server in Java
https://cantaloupe-project.github.io/
Other
266 stars 105 forks source link

Exception: java.lang.IllegalArgumentException: Invalid ICC Profile Data #664

Open anuroopkancherla opened 3 weeks ago

anuroopkancherla commented 3 weeks ago

VERSION= Cantaloupe 5.0.5

processor.selection_strategy = AutomaticSelectionStrategy

URL: /iiif/3/<>/info.json

We are using Cantaloupe 5.0.5 server version to process the images. We use AutomaticSelectionStrategy to process the jpg images and while processing some of the jpg images we are observing the Invalid ICC Profile Data exception and the image is not rendering.

500 Internal Server Error

Invalid ICC Profile Data (through reference chain: edu.illinois.library.cantaloupe.processor.codec.jpeg.JPEGMetadata["exif"])

com.fasterxml.jackson.databind.JsonMappingException: Invalid ICC Profile Data (through reference chain: edu.illinois.library.cantaloupe.processor.codec.jpeg.JPEGMetadata["exif"]) at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:397) at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:356) at com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.java:316) at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:731) at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:166) at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) at com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3024) at com.fasterxml.jackson.core.base.GeneratorBase.writeObject(GeneratorBase.java:388) at com.fasterxml.jackson.core.JsonGenerator.writeObjectField(JsonGenerator.java:1838) at edu.illinois.library.cantaloupe.image.InfoSerializer.serialize(InfoSerializer.java:56) at edu.illinois.library.cantaloupe.image.InfoSerializer.serialize(InfoSerializer.java:16) at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1513) at com.fasterxml.jackson.databind.ObjectWriter._configAndWriteValue(ObjectWriter.java:1215) at com.fasterxml.jackson.databind.ObjectWriter.writeValueAsString(ObjectWriter.java:1085) at edu.illinois.library.cantaloupe.image.Info.toJSON(Info.java:513) at edu.illinois.library.cantaloupe.processor.AbstractImageIOProcessor.readInfo(AbstractImageIOProcessor.java:104) at edu.illinois.library.cantaloupe.cache.InfoService.readInfo(InfoService.java:220) at edu.illinois.library.cantaloupe.cache.InfoService.getOrReadInfo(InfoService.java:153) at edu.illinois.library.cantaloupe.cache.CacheFacade.getOrReadInfo(CacheFacade.java:67) at edu.illinois.library.cantaloupe.resource.AbstractRequestHandler.getOrReadInfo(AbstractRequestHandler.java:39) at edu.illinois.library.cantaloupe.resource.InformationRequestHandler.handle(InformationRequestHandler.java:287) at edu.illinois.library.cantaloupe.resource.iiif.v3.InformationResource.doGET(InformationResource.java:85) at edu.illinois.library.cantaloupe.resource.HandlerServlet.handle(HandlerServlet.java:97) at edu.illinois.library.cantaloupe.resource.HandlerServlet.doGet(HandlerServlet.java:35) at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:550) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.IllegalArgumentException: Invalid ICC Profile Data at java.desktop/sun.java2d.cmm.ProfileDataVerifier.verify(ProfileDataVerifier.java:56) at java.desktop/java.awt.color.ICC_Profile.getInstance(ICC_Profile.java:773) at java.desktop/com.sun.imageio.plugins.jpeg.JFIFMarkerSegment$ICCMarkerSegment.getNativeNode(JFIFMarkerSegment.java:1545) at java.desktop/com.sun.imageio.plugins.jpeg.JFIFMarkerSegment.getNativeNode(JFIFMarkerSegment.java:240) at java.desktop/com.sun.imageio.plugins.jpeg.JPEGMetadata.getNativeTree(JPEGMetadata.java:785) at java.desktop/com.sun.imageio.plugins.jpeg.JPEGMetadata.getAsTree(JPEGMetadata.java:757) at edu.illinois.library.cantaloupe.processor.codec.IIOMetadata.getAsTree(IIOMetadata.java:24) at edu.illinois.library.cantaloupe.processor.codec.jpeg.JPEGMetadata.getEXIF(JPEGMetadata.java:39) at jdk.internal.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:689) at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:723) ... 51 more

Please help us to understand the problem and potential workaround to solve this exception and also want to understand if this issue is related to cantaloupe server.

Thank you !!!!

VonUniGE commented 3 weeks ago

The latest version of Cantaloupe is 5.0.6. Could you please provide a test case that reproduces this error using version 5.0.6?

DiegoPino commented 3 weeks ago

@anuroopkancherla Java2D (the processor that was automatically selected) does an internal ICC validation (see https://code.googlesource.com/edge/openjdk/+/refs/heads/master/jdk/src/share/classes/sun/java2d/cmm/ProfileDataVerifier.java) and when that fails it throws an exception. As @VonUniGE correctly stated, without seeing an example of an affected image it is hard determine if Java2D is not being able to recognize a valid color profile or the color profile is indeed wrong (and we have seen those in the wild many times).

There might be a valid point of discussion/to be made if an exception at that level should end in a 500 code (and break the whole processing chain) or should instead generate log warnings (which maybe people won't read...) and still attempt to process even if EXIF processing failed.

Could you also try processing JPEGs with one of the other processors to compare? (can be easily changed in the settings)

anuroopkancherla commented 3 weeks ago

@VonUniGE @DiegoPino As per your suggestions I have tried with 5.0.6 version as well, I am still seeing same issue.

Output Logs:

2024-07-03 12:38:55 16:38:55.470 [qtp2089016471-15] INFO e.i.l.c.r.i.v.InformationResource - Handling GET /iiif/3/603002_Box15_Folder05-001.jpg/info.json 2024-07-03 12:38:55 16:38:55.471 [qtp2089016471-15] DEBUG e.i.l.c.r.i.v.InformationResource - Request headers: Cookie: _ga=GA1.1.475889717.1717215540; _ga_0VS4QBSPD8=GS1.1.1719506661.11.0.1719506661.0.0.0; Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.7; Connection: keep-alive; User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36; Sec-Fetch-Site: none; Sec-Fetch-Dest: document; Host: localhost:8182; Accept-Encoding: gzip, deflate, br, zstd; Sec-Fetch-Mode: navigate; sec-ch-ua: "Not/A)Brand";v="8", "Chromium";v="126", "Google Chrome";v="126"; sec-ch-ua-mobile: ?0; Upgrade-Insecure-Requests: 1; sec-ch-ua-platform: "Windows"; Sec-Fetch-User: ?1; Accept-Language: en-US,en;q=0.9 2024-07-03 12:38:55 16:38:55.472 [qtp2089016471-15] DEBUG e.i.l.c.i.MetaIdentifier - [Raw path component: 603002_Box15_Folder05-001.jpg] -> [decoded: 603002_Box15_Folder05-001.jpg] -> [slashes substituted: 603002_Box15_Folder05-001.jpg] 2024-07-03 12:38:55 16:38:55.670 [qtp2089016471-15] DEBUG s.a.a.requestId - Received failed response: 404, Request ID: N2JKRDYYM06A67EA, Extended Request ID: D4gX9MPD5tzGc72bt6W3VFzNOHIcSEjw1ttim1HJrMHxIOx4mXYnTo0TuxVA4IywqEO6tYpYk7PFBs1oMn1Nh80jWOrQqj2DhVlGA9tCjnM= 2024-07-03 12:38:55 16:38:55.674 [qtp2089016471-15] DEBUG e.i.l.c.c.FilesystemCache - getSourceImageFile(): C:\Users\anuro\Desktop\Work Zone\NextGen NAC Local Setup\catalog-infrastructure\ngc-cantaloupe\cantaloupe\cache/source/08/e7/fd/08e7fd7ef075a4a9a0d6225d9d9e939c 2024-07-03 12:38:55 16:38:55.675 [qtp2089016471-15] DEBUG e.i.l.c.s.FilesystemSource - Resolved 603002_Box15_Folder05-001.jpg to images/603002_Box15_Folder05-001.jpg 2024-07-03 12:38:55 16:38:55.675 [qtp2089016471-15] DEBUG e.i.l.c.p.ProcessorFactory - Failed to initialize TurboJpegProcessor (error: /opt/libjpeg-turbo/lib/libturbojpeg.so: /opt/libjpeg-turbo/lib/libturbojpeg.so: file too short) 2024-07-03 12:38:55 16:38:55.675 [qtp2089016471-15] DEBUG e.i.l.c.p.ProcessorFactory - Java2dProcessor selected for format JPEG (AutomaticSelectionStrategy offered TurboJpegProcessor, Java2dProcessor) 2024-07-03 12:38:55 16:38:55.675 [qtp2089016471-15] DEBUG e.i.l.c.p.ProcessorConnector - File -> FileProcessor connection between FilesystemSource and Java2dProcessor 2024-07-03 12:38:55 16:38:55.705 [qtp2089016471-15] DEBUG s.a.a.requestId - Received failed response: 404, Request ID: N2JKMED0SNXN91M4, Extended Request ID: PPXo6MTdKB72/MwDqC8xtklw5KHIn8N8PM316KtsNHYVwxI96+jF+3umg4uNW0yHopj27uENJY1b7aoLQhkZDBJyAyxX0sucr5Ed4mituIQ= 2024-07-03 12:38:55 16:38:55.707 [qtp2089016471-15] TRACE e.i.l.c.p.c.j.JPEGImageReader - ImageIO plugin preferences: com.sun.imageio.plugins.jpeg.JPEGImageReader 2024-07-03 12:38:55 16:38:55.707 [qtp2089016471-15] DEBUG e.i.l.c.p.c.j.JPEGImageReader - Using com.sun.imageio.plugins.jpeg.JPEGImageReader 2024-07-03 12:38:55 16:38:55.730 [qtp2089016471-15] INFO e.i.l.c.r.ErrorResource - Handling GET /iiif/3/603002_Box15_Folder05-001.jpg/info.json 2024-07-03 12:38:55 16:38:55.731 [qtp2089016471-15] DEBUG e.i.l.c.r.ErrorResource - Request headers: Cookie: _ga=GA1.1.475889717.1717215540; _ga_0VS4QBSPD8=GS1.1.1719506661.11.0.1719506661.0.0.0; Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.7; Connection: keep-alive; User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36; Sec-Fetch-Site: none; Sec-Fetch-Dest: document; Host: localhost:8182; Accept-Encoding: gzip, deflate, br, zstd; Sec-Fetch-Mode: navigate; sec-ch-ua: "Not/A)Brand";v="8", "Chromium";v="126", "Google Chrome";v="126"; sec-ch-ua-mobile: ?0; Upgrade-Insecure-Requests: 1; sec-ch-ua-platform: "Windows"; Sec-Fetch-User: ?1; Accept-Language: en-US,en;q=0.9 2024-07-03 12:38:55 16:38:55.731 [qtp2089016471-15] DEBUG e.i.l.c.r.ErrorResource - Base URI assembled from X-Forwarded headers: http://localhost:8182 2024-07-03 12:38:55 16:38:55.737 [qtp2089016471-15] DEBUG e.i.l.c.r.HandlerServlet - Responded to GET /iiif/3/603002_Box15_Folder05-001.jpg/info.json with HTTP 500 in 267 msec 2024-07-03 12:38:55 172.20.0.1 - - [03/Jul/2024:16:38:55 +0000] "GET /iiif/3/603002_Box15_Folder05-001.jpg/info.json HTTP/1.1" 500 6765 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"

Below is my Processor Selection in my properties file, I have tried with all the combination of processors for JPG but non has worked out to resolve my issue

----------------------------------------

Processor Selection

----------------------------------------

processor.selection_strategy = AutomaticSelectionStrategy

processor.ManualSelectionStrategy.avi = FfmpegProcessor processor.ManualSelectionStrategy.bmp = processor.ManualSelectionStrategy.flv = FfmpegProcessor processor.ManualSelectionStrategy.gif = processor.ManualSelectionStrategy.jp2 = KakaduNativeProcessor processor.ManualSelectionStrategy.jpg = processor.ManualSelectionStrategy.mov = FfmpegProcessor processor.ManualSelectionStrategy.mp4 = FfmpegProcessor processor.ManualSelectionStrategy.mpg = FfmpegProcessor processor.ManualSelectionStrategy.pdf = PdfBoxProcessor processor.ManualSelectionStrategy.png = processor.ManualSelectionStrategy.tif = processor.ManualSelectionStrategy.webm = FfmpegProcessor processor.ManualSelectionStrategy.xpm =

Fall back to this processor for any formats not assigned above. processor.ManualSelectionStrategy.fallback = Java2dProcessor

I cannot share the sample image but below is the metadata of the image which is causing the issue.

ExifTool Version Number : 12.76 File Name : 603002_Box15_Folder05-001.jpg Directory : . File Size : 1444 kB File Modification Date/Time : 2023:08:14 20:35:43+00:00 File Access Date/Time : 2024:07:03 16:22:21+00:00 File Inode Change Date/Time : 2024:07:03 16:21:13+00:00 File Permissions : -rw-r--r-- File Type : JPEG File Type Extension : jpg MIME Type : image/jpeg JFIF Version : 1.01 Resolution Unit : inches X Resolution : 300 Y Resolution : 300 Exif Byte Order : Big-endian (Motorola, MM) Orientation : Horizontal (normal) Software : Microsoft Windows Photo Viewer 6.1.7600.16385 Modify Date : 2018:01:29 10:57:45 Padding : (Binary data 2060 bytes, use -b option to extract) Warning : Bad length ICC_Profile (length 669051) About : uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b Creator Tool : Microsoft Windows Photo Viewer 6.1.7600.16385 Image Width : 4488 Image Height : 2968 Encoding Process : Baseline DCT, Huffman coding Bits Per Sample : 8 Color Components : 3 Y Cb Cr Sub Sampling : YCbCr4:2:0 (2 2) Image Size : 4488x2968 Megapixels : 13.3

jcoyne commented 1 week ago

Did you see?

 Warning : Bad length ICC_Profile (length 669051)