RuntimeException: XMP data size exceededs JPEG segment size

[sinedsem]

Hi @dragon66! It's me again from stockhelper again =)

As usual, file : https://github.com/sinedsem/test/blob/master/7.jpg

Stack trace:

java.lang.RuntimeException: XMP data size exceededs JPEG segment size at com.icafe4j.image.meta.jpeg.JpegXMP.write(Unknown Source) at com.icafe4j.image.jpeg.JPEGTweaker.insertXMP(Unknown Source) at com.icafe4j.image.jpeg.JPEGTweaker.insertXMP(Unknown Source) at com.icafe4j.image.meta.Metadata.insertXMP(Unknown Source)

P. S. Please find another glass of beer in your PayPal wallet =)

[dragon66]

Hi sinedsem,

Unfortunately you have hit one of the limit of icafe.

This is how it happens: for JPEG image, each metadata block could hold about 65536 bytes of data. When it comes to XMP, if the total size in bytes is within the rough limit of 65536, it will work automatically and insert that piece of data into the image. Otherwise, what should be done is the ability to split the large data trunk into separate 65536 block of data.

Adobe specification specifies that the entire XMP split into two parts: first, the normal XMP, and second, the ExtendedXMP. The first part is a normal XMP package which is able to fit into one block of the image while the second part - ExtendedXMP could be any size and later, needs to be split into smaller trunks with each of the trunk fits into one block of the image.

That said, icafe at the present stage, has no way to split the XMP data according to the specification and if it finds that the XMP data cannot be squeezed into one block of the image, it will throw an exception which is the error message you saw.

I can't think about a way to do this automatically in icafe now if that's what you want. I haven't checked if the Adobe XMP API for Java could do this or not. If it has this ability, you may leverage that and split the XMP data before inserting using icafe.

Or I guess there could be a workaround like this: you create a placeholder XMP normal part without any actual data in it (with the necessary artifacts). Then all the actual XMP data, you can pass as ExtendedXMP to icafe.

I haven't tried this myself though but you can give it a shot.

When I have a chance, I will try this too and if it works, add this workaround to icafe as a temporary fix for this issue. Will let you know if I can make it work.

By the way, thanks a lot for the beer!

Wen

[sinedsem]

@dragon66, yes, now I see, this image already has a lot of xmp data.

So, what should I suggest to the author of this pic? Manually clear unneccessary tags?

Are you going to fix this in future?

[dragon66]

@sinedsem It doesn't matter whether or not there is already XMP in the image before inserting new XMP. ICafe will remove the old data and insert new data. In your case, I guess the user tried to insert an XMP data trunk which is larger than one JPEG image segment can hold. Are you able to obtain the actual XMP data from the user who failed to insert it into the image or if that is not possible ask the user how big the data is when converted to binary?

I will take a look and see how far I can go in fixing this issue.

[sinedsem]

@dragon66 I think we can use XMPUtils.packageForJPEG() from com.adobe.xmp.xmpcore library. I will investigate more...

[sinedsem]

Ok, I've figured out how to work with extended xmp:

Map<MetadataType, Metadata> metadataMap = Metadata.readMetadata(source);

XMPMeta xmpMeta;
if (metadataMap != null && metadataMap.get(MetadataType.XMP) != null) {
    xmpMeta = XMPMetaFactory.parseFromBuffer(metadataMap.get(MetadataType.XMP).getData());
} else {
    xmpMeta = new XMPMetaImpl();
}

// edit xmpMeta

List<Metadata> values = metadataMap != null ? new ArrayList<>(metadataMap.values()) : new ArrayList<>();

StringBuilder xmp = new StringBuilder();
StringBuilder extXmp = new StringBuilder();

XMPUtils.packageForJPEG(xmpMeta, xmp, extXmp, new StringBuilder());
if (extXmp.length() > 0) {
    values.add(new JpegXMP(xmp.toString(), extXmp.toString()));
} else {
    values.add(new JpegXMP(xmp.toString()));
}
Metadata.insertMetadata(values, source, destination);
destination.close();

this requires com.adobe.xmp.xmpcore library:

<dependency>
   <groupId>com.adobe.xmp</groupId>
   <artifactId>xmpcore</artifactId>
   <version>6.1.10</version>
</dependency>

[dragon66]

@sinedsem I changed a bit of the JpegXMP class to auto split the XMP data into standard and extended XMP if the size is beyond the limit of one JPEG segment. This is transparent to user. Now you can do directly insert any size of XMP without using corresponding Adobe XMP core function.