InvalidClassException during FlexCompRowMatrix deserialization

[AlexanderScherbatiy]

We use the HTM to detect anomalies in our Spark Streaming process.

After running our project the following exception arises during the deserialization: java.io.InvalidClassException: no.uib.cipr.matrix.sparse.FlexCompRowMatrix; no valid constructor

It seems that the SDRClassifier uses FlexCompRowMatrix class and the FlexCompRowMatrix extends AbstractMatrix class which neither implements Serializable interface nor has a constructor with no-arg arguments.

Is it possible to do something to avoid the InvalidClassException in our case?

The used version is org.numenta/htm.java 0.6.13.

See also discussion: https://discourse.numenta.org/t/sdrclassifier-serialization/3505

[cogmission]

@AlexanderScherbatiy Hi,

...and thank you for using HTM.Java! I'm looking into this now...

[cogmission]

Hi @alexsch,

The problem is that the FlexCompRowMatrix class we use comes from a "culled" and highly optimized version which I forked from AlgorithmFoundry - hence the name, "algorithmfoundry-shade-culled-1.x.jar" for the library name (which is included in the "libs" directory of the project). It adds the ability to delete rows and columns from the matrix class. I wanted to expand its ability to work with sparse matrices, since this library offers many advanced features needed for other things like the KNN Classifier development work.

We made this available via Gradle or Maven via the repository on my server using the following maven url: http://metaware.us/maven3

...as seen in the Gradle "build.gradle" file:

https://github.com/numenta/htm.java/blob/master/build.gradle#L20

The only reason this might not work for you is that you are specifying another algorithmfoundry repository and picking up the wrong version of the class?? You have to make sure you are using our version...

Let me know if this helps?

[AlexanderScherbatiy]

We do not use additional repositories.

The dependency on the numenta library in our pom file is:

<dependency>
    <groupId>org.numenta</groupId>
    <artifactId>htm.java</artifactId>
    <version>0.6.13</version>
</dependency>

The dependecy:tree shows the algorithmfoundry library only in one place:

[INFO] +- org.numenta:htm.java:jar:0.6.13:compile
[INFO] |  +- joda-time:joda-time:jar:2.5:compile
[INFO] |  +- com.chaschev:chutils:jar:1.4:compile
[INFO] |  +- net.sf.trove4j:trove4j:jar:3.0.3:compile
[INFO] |  +- io.reactivex:rxjava:jar:1.0.10:compile
[INFO] |  +- com.cedarsoftware:java-util:jar:1.19.3:compile
[INFO] |  |  +- org.apache.logging.log4j:log4j-api:jar:2.1:compile
[INFO] |  |  \- org.apache.logging.log4j:log4j-core:jar:2.1:compile
[INFO] |  +- de.ruedigermoeller:fst:jar:2.45:compile
[INFO] |  |  +- org.javassist:javassist:jar:3.19.0-GA:compile
[INFO] |  |  \- org.objenesis:objenesis:jar:2.1:compile
[INFO] |  +- org.openjdk.jmh:jmh-core:jar:1.11.3:compile
[INFO] |  \- algorithmfoundry:algorithmfoundry-shade-culled:jar:1.3:compile

It is the algorithmfoundry:algorithmfoundry-shade-culled:jar:1.3:compile