Upgrade to JEP 4.2.0

[aberenguel]

Using Ubuntu 24.04 with latest IPED (master d436a740d), profile pedo.

The error occurred in the end of processing.

2024-06-28 03:26:28     [INFO]  [engine.core.Manager]                   Changed to processing queue with priority 1
2024-06-28 03:26:28     [INFO]  [engine.data.IPEDSource]                        Opening index /xxxxxxxxx/iped/index
2024-06-28 03:26:51     [INFO]  [engine.data.IPEDSource]                        Index opened
2024-06-28 03:26:51     [INFO]  [engine.data.IPEDSource]                        Creating LuceneId to ID mapping...
java.lang.RuntimeException: double[]
        at iped.engine.data.IPEDSource.<init>(IPEDSource.java:252)
        at iped.engine.data.IPEDSource.<init>(IPEDSource.java:161)
        at iped.engine.search.ItemSearcher.<init>(ItemSearcher.java:29)
        at iped.engine.core.Manager.monitorProcessing(Manager.java:622)
        at iped.engine.core.Manager.process(Manager.java:298)
        at iped.app.processing.Main.startManager(Main.java:178)
        at iped.app.processing.Main.execute(Main.java:240)
        at iped.app.processing.Main.main(Main.java:308)
Caused by: java.lang.ClassNotFoundException: double[]
        at java.base/java.lang.Class.forName0(Native Method)
        at java.base/java.lang.Class.forName(Class.java:315)
        at iped.engine.task.index.IndexItem.loadMetadataTypes(IndexItem.java:220)
        at iped.engine.data.IPEDSource.<init>(IPEDSource.java:236)
        ... 7 more
2024-06-28 03:26:57     [ERROR] [app.processing.Main]                   Processing Error: 
java.lang.RuntimeException: double[]
        at iped.engine.data.IPEDSource.<init>(IPEDSource.java:252) ~[iped-engine-4.2-snapshot.jar:?]
        at iped.engine.data.IPEDSource.<init>(IPEDSource.java:161) ~[iped-engine-4.2-snapshot.jar:?]
        at iped.engine.search.ItemSearcher.<init>(ItemSearcher.java:29) ~[iped-engine-4.2-snapshot.jar:?]
        at iped.engine.core.Manager.monitorProcessing(Manager.java:622) ~[iped-engine-4.2-snapshot.jar:?]
        at iped.engine.core.Manager.process(Manager.java:298) ~[iped-engine-4.2-snapshot.jar:?]
        at iped.app.processing.Main.startManager(Main.java:178) [iped.jar:?]
        at iped.app.processing.Main.execute(Main.java:240) [iped.jar:?]
        at iped.app.processing.Main.main(Main.java:308) [iped.jar:?]
Caused by: java.lang.ClassNotFoundException: double[]
        at java.lang.Class.forName0(Native Method) ~[?:?]
        at java.lang.Class.forName(Class.java:315) ~[?:?]
        at iped.engine.task.index.IndexItem.loadMetadataTypes(IndexItem.java:220) ~[iped-engine-4.2-snapshot.jar:?]
        at iped.engine.data.IPEDSource.<init>(IPEDSource.java:236) ~[iped-engine-4.2-snapshot.jar:?]
        ... 7 mor

[aberenguel]

Trying to continue with --continue leads to the same error.

[wladimirleite]

Does it work with IPED 4.1.x? Can you share the image with me so I can take a closer look?

[aberenguel]

I investigated and found that the file iped/conf/metadataTypes.txt in the case has the following line:

face_encodings = double[]

[aberenguel]

Does it work with IPED 4.1.x?

I haven't tested yet.

Can you share the image with me so I can take a closer look?

The image has 900 GB. Should I share locally with you?

[wladimirleite]

I investigated and found that the file iped/conf/metadataTypes.txt in the case has the following line:

face_encodings = double[]

Try to remove this line. I don't see it in my local version of this file, neither in the master branch.

[aberenguel]

Some tests in Java:

System.out.println(double[].class.getName()); // outputs "[D"
System.out.println(double[].class.getCanonicalName());  // outputs "double[]"

Class.forName("[D"); // returns double[]
Class.forName("double[]"); // throws ClassNotFoundException

[aberenguel]

I investigated and found that the file iped/conf/metadataTypes.txt in the case has the following line:

face_encodings = double[]

Try to remove this line. I don't see it in my local version of this file, neither in the master branch.

This line is only in the metadataTypes.txt file of the case. I set enableFaceRecognition = true.

[aberenguel]

I think the bug can be here: https://github.com/sepinf-inc/IPED/blob/d436a740da72e56d87da6b6bdc1b77c4ddbff156/iped-engine/src/main/java/iped/engine/task/index/IndexItem.java#L208

[aberenguel]

I investigated and found that the file iped/conf/metadataTypes.txt in the case has the following line:

face_encodings = double[]

Try to remove this line. I don't see it in my local version of this file, neither in the master branch.

I replaced by face_encodings = [D and worked!

[wladimirleite]

There are no arrays in metadataTypes.txt. I guess it should either ignore (not save in the file) array values, or set it as the corresponding object wrapper class of the array elements (in this case java.lang.Double).

[lfcnassif]

face_encodings = double[]

Arrays shouldn't appear into metadataTypes.txt.

or set it as the corresponding object wrapper class of the array elements (in this case java.lang.Double).

I think this is the best approach.

[lfcnassif]

I'll try to reproduce the issue here, I think it happens when the first image where a face is detected have 2 or more faces. If the first image has 1 face, it works with later images with 2+ faces.

[wladimirleite]

This line is only in the metadataTypes.txt file of the case. I set enableFaceRecognition = true.

@aberenguel, is it an "--append"?

[lfcnassif]

@aberenguel what is the jep python lib version you installed on Linux?

[aberenguel]

--append

No. It is the first evidence.

[lfcnassif]

I processed a case with a single image with 2 faces and I didn't reproduce. The mapping into conf/metadataTypes.txt is: face_encodings = jep.NDArray

Not sure from where that double[] is coming from...

[aberenguel]

@aberenguel what is the jep python lib version you installed on Linux?

I'm using Python virtual environments. It used jep 4.0.3: ./lib/python3.12/site-packages/jep/jep-4.0.3.jar

[lfcnassif]

You didn't change FaceRecognitionTask.py code, right?

[aberenguel]

You didn't change FaceRecognitionTask.py code, right?

No. It was not modified.

[aberenguel]

I wonder if it is thing related to Python 3.12.

[aberenguel]

Processing a folder with this single file triggers this issue.

[lfcnassif]

I wasn't able to reproduce on Windows with image above, IPED detected 7 faces in that image with the correct type: face_encodings = jep.NDArray

Please @aberenguel try to reproduce on Linux with python 3.9 and numpy 1.x (not 2.0), the same used on Windows.

[aberenguel]

Please @aberenguel try to reproduce on Linux with python 3.9 and numpy 1.x (not 2.0), the same used on Windows.

Ubuntu 24.04 has only python 3.12. I'll try to get python 3.9 in other way. Btw, the numpy is version 1.26.4.

[lfcnassif]

Try to use conda, you can install different python versions with it.

[lfcnassif]

@aberenguel have you confirmed this happens because of python 3.12? Does it work with previous python versions?

[aberenguel]

@aberenguel have you confirmed this happens because of python 3.12? Does it work with previous python versions?

I've just tested with Python 3.9.19 (fresh installing modules jep==4.0.3, tensorflow, face_recognition and bs4). The error was the same.

I get some old cases I processed in Ubuntu 22.04 and they have face_encodings = jep.NDArray. So I think it is not related to Python version.

[aberenguel]

Another test I've just done. I upgraded the jep to version 4.2.0 (in pom.xml and with pip). There is no error anymore and the case was generated with face_encodings = jep.NDArray.

[lfcnassif]

I upgraded the jep to version 4.2.0 (in pom.xml and with pip).

Great! I was just going to ask you to try that :-)

[aberenguel]

Confirmed that JEP 4.2.0 solved the problem with Python 3.9 and 3.12

[patrickdalla]

Great, @aberenguel , thank you for testing. JEP 4.2 has some important improvements, mainly "Java Methods can be Annotated to accept Python kwargs", that will be very useful in Aleapp integration. I have made some workaround code for ALeapp bridge task working with the old JEP version, but I think I can remove them with JEP 4.2 and make a cleaner and less complex integration code.