search

sepinf-inc / IPED

IPED Digital Forensic Tool. It is an open source software that can be used to process and analyze digital evidence, often seized at crime scenes by law enforcement or in a corporate investigation by private examiners.
Other
952 stars 218 forks source link

Evaluate disabling some JAI ImageIO plugins #1134

Open lfcnassif opened 2 years ago

lfcnassif commented 2 years ago

I've just fixed some "cannot seek" exceptions being thrown in #1116 (3474322) caused by JAI ImageIO TIFFImageReader being used in that branch. On current master, the internal Sun impl is used and the exceptions are not thrown while processing, but when opening UI, the JAI impl is used and causes exceptions to be thrown, making behavior inconsistent (this took me some time to debug...).

Java 9 added support for TIFF. JAI plugin has some intersections with formats already supported by the JVM, I think we should evaluate which implementations are better and disable the others, so the dynamic service loader behavior won't change results...

lfcnassif commented 2 years ago

@tc-wleite, out of curiosity, have you done some related study in the past? (I have a far remind about that...)

wladimirleite commented 2 years ago

@tc-wleite, out of curiosity, have you done some related study in the past? (I have a far remind about that...)

Well, I don't remember :-) I did some comparisons, but I think they didn't involve JAI. And definitely didn't test with Java >= 9.

That being said, I think the intersection you mentioned is a valid concern. Removing JAI altogether would be great, but probably won't cover all the formats, right?

lfcnassif commented 2 years ago

but probably won't cover all the formats, right?

Yes, they support pcx, pnm & raw, not supported by the JVM, and even supported formats could have different codec support...

I've also seen NPEs being thrown when parsing BMP files for a while. I'll do some testing to check which plugin is the bad guy, and if the other handles the triggering files fine. But switching would require a larger test...

lfcnassif commented 2 years ago

All 1147 NPEs (from BMPs in the 33M dataset) comes from JAI. Removing JAI from classpath causes different IOExceptions to be thrown by the internal JVM BMP image reader, from all those BMPs. All of them are corrupted. Although NPEs commonly are symptoms of bugs, this corrupted BMP issue doesn't seem critical to me.

PS: Tika 2 causes JAI impls to take priority, maybe they changed something in service loading...

lfcnassif commented 1 year ago

Just hit some NPEs running regression tests while processing (corrupted) BMPs again, caused by JAI. Today we have 3 BMP decoder providers: JAI, the JVM default and TwelveMonkeys. My concern is that ImageIO service loading seems to be not deterministic, linking with TSK-4.11.1 those exceptions weren't thrown, but linking with TSK-4.12.0 they were...

lfcnassif commented 1 year ago

I could confirm the exceptions stopped to be thrown excluding JAI from classpath. As all those BMPs are corrupted and doesn't render, I think this is not critical, but we should have a deterministic behavior...