Cannot be serialized

[cbobed]

Hi,

it seems that version 4.0.3 cannot be serialized due to ProcessMonitors (they seem not to be marked as serializable). I've tried using directly the tools that java provides for serialization, as well as apache-commons serialization-utils (both object and serialization utils).

Cheers,

C.

[cbobed]

Hi,

I've tried versions so far 1.* and 4.* with no luck, I was about to try version 5, but the current pom.xml dependencies seems to be broken:

[ERROR] Failed to execute goal on project jfact: Could not resolve dependencies for project net.sourceforge.owlapi:jfact:bundle:5.0.0: Could not find artifact net.sourceforge.owlapi:owlapi-distribution:jar:5.0.2-SNAPSHOT in sonatype-nexus-snapshots (https://oss.sonatype.org/content/repositories/snapshots) -> [Help 1]

cheers, C.

[cbobed]

Hi,

substituting the OWLAPI dependency with the OWLAPI 5.0.3 solves the dependency problem. However, I've just tested also JFact 5 (just in case it could be the solution), but it seems that it does not support serialization either:

java.io.NotSerializableException: uk.ac.manchester.cs.jfact.kernel.ExpressionCache$$Lambda$112/1535116392

I've used Java 1.8.0_65

Cheers,

C.

[ignazio1977]

Yes - looks like lambdas and serialization do not agree when lambdas are stored as class members. I'm trying to find a way around it.

[ansell]

Does setting them to be transient work? Hopefully you don't need to override serialisation just to support it.

[ignazio1977]

It has worked to avoid the obvious errors; I'm still not being able to deserialise a working reasoner.

Trying a different serialization mechanism might be worth it - XStream, for example.

[cbobed]

I would try using Perst or db4o oodbs, but they don't support Java 1.8 as far as I know. Could Jfact 4 be serialized with the modifications you have made?

[cbobed]

I've tried to serialize the version 4 using Xstream. I've tried both making ExpressionCache serializable, and marking all its instances as transient. While it is able to serialize the reasoner, it raises a nullPointerException when deserializing the instance using Java serialization, Apache commons, and XStream. They fail in the same point (I attach the debug information Xstream has output just in case it helps).

Cheers,

C

---- Debugging information ---- cause-exception : java.lang.NullPointerException cause-message : null class : java.util.LinkedHashSet required-type : java.util.LinkedHashSet converter-type : com.thoughtworks.xstream.converters.collections.CollectionConverter path : /uk.ac.manchester.cs.jfact.JFactReasoner/root/delegate/uk.ac.manchester.cs.owl.owlapi.OWLImmutableOntologyImpl/default/manager/uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl/default/importsClosureCache/entry/linked-hash-set/uk.ac.manchester.cs.owl.owlapi.concurrent.ConcurrentOWLOntologyImpl line number : 99149 class[1] : java.util.concurrent.ConcurrentHashMap

converter-type[1] : com.thoughtworks.xstream.converters.collections.MapConverter

at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:79)
at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:70)
at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66)
at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:50)
at com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.readItem(AbstractCollectionConverter.java:73)
at com.thoughtworks.xstream.converters.collections.MapConverter.putCurrentEntryIntoMap(MapConverter.java:110)
at com.thoughtworks.xstream.converters.collections.MapConverter.populateMap(MapConverter.java:98)
at com.thoughtworks.xstream.converters.collections.MapConverter.populateMap(MapConverter.java:92)
at com.thoughtworks.xstream.converters.collections.MapConverter.unmarshal(MapConverter.java:87)
at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72)
at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:70)
at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66)
at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:50)
at com.thoughtworks.xstream.converters.reflection.SerializableConverter$2.defaultReadObject(SerializableConverter.java:402)
at com.thoughtworks.xstream.core.util.CustomObjectInputStream.defaultReadObject(CustomObjectInputStream.java:119)
at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.readObject(OWLOntologyManagerImpl.java:1399)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.thoughtworks.xstream.core.util.SerializationMembers.callReadObject(SerializationMembers.java:125)
... 32 more

Caused by: java.lang.NullPointerException at uk.ac.manchester.cs.owl.owlapi.concurrent.ConcurrentOWLOntologyImpl.hashCode(ConcurrentOWLOntologyImpl.java:81) at java.util.HashMap.hash(HashMap.java:338) at java.util.HashMap.put(HashMap.java:611) at java.util.HashSet.add(HashSet.java:219) at com.thoughtworks.xstream.converters.collections.CollectionConverter.addCurrentElementToCollection(CollectionConverter.java:99) at com.thoughtworks.xstream.converters.collections.CollectionConverter.populateCollection(CollectionConverter.java:91) at com.thoughtworks.xstream.converters.collections.CollectionConverter.populateCollection(CollectionConverter.java:85) at com.thoughtworks.xstream.converters.collections.CollectionConverter.unmarshal(CollectionConverter.java:80) at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72) ... 52 more

[cbobed]

I've managed to serialize and deserialize JFact 1.2.4 with owlapi 3.5.4 by modifying the hashCode function of OWLOntologyImpl (it seems that the problem was that when deserializing it expects some elements in particular positions, and it neglects the overriden hash function). However, I'm not sure about the implications of such change in the OWLAPI.

[cbobed]

New advances and a potential bug,

I've moved to OWLAPI 4.2.6 and JFact 4.0.3. Marking kernel.ExpressionCache as Serializable allows to clone a reasoner (if you are working with a OWLAPI version which allows serialization - I've just opened an issue in the OWLAPI project). However, the cloned reasoner seems to have problems with some predefined OWLClass expressions. When asked for the classes equivalent to OWLThing or OWLNothing for example, the Node returned has exactly the same size in both the original and cloned reasoner instances, but OWLThing/OWLNothing references are substituted by null pointers which makes it raise some errors while materializing some inferences.

Cheers,

Carlos

[ignazio1977]

The disappearance of owl:Nothing is due to the lack of equals()/hashCode() on ConceptBottom. The deserialized reasoner has new instances for the singletons representing top and bottom, and they're not found in the reverse mapping hashmap for this reason.

Fixed that, the next issue is that modifying the root ontology does not seem to have an effect - adding an unsatisfiable class does not result in a new entry in the classes equivalent to bottom.

[ignazio1977]

The deserialized reasoner is not notified of changes - it needs to register itself as a listener again, probably.

[cbobed]

I usually work in a buffered way, and I don't recall having such problem (not being aware of added axioms). This might be consistent with the fact that you comment. I'll check it as soon as possible.

Is it fixed in the current head of version 4?

[ignazio1977]

Fixed - it will require a new OWLAPI 4 release though.

[ignazio1977]

ff13b58e520d15e381d8f5b5c6be6e140e2dd363

[ignazio1977]

Yes it's in version 4 now - I haven't released it yet.

The issue happens on deserialized reasoners only. The list of ontology change listeners in OWLOntologyManager is transient, so it does not contain anything after a deserialization - correctly, as we cannot consider listening objects as still valid in general (it's usually not OWLAPI components). Custom readObject is enough to fix this.

[ignazio1977]

OWLAPI 4.2.7 and JFact 4.0.4 have been released.

[ignazio1977]

Fixed in version 5 as well. Released JFact 5.0.1

[cbobed]

Hi,

I've been the latest 4.* version, but when I'm trying to retrieve all the inferred axioms, it crashes: The code: System.out.println(retrieveAllInferredAxioms(baseReasoner).size()); System.out.println(retrieveAllInferredAxioms(clonedReasoner).size());

with retrieveAllInferredAxioms(reasoner) being as follows:

public static Set retrieveAllInferredAxioms (OWLReasoner reasoner) { Set results = new HashSet<>();
InferredClassAssertionAxiomGenerator a = new InferredClassAssertionAxiomGenerator(); results.addAll(a.createAxioms(reasoner.getRootOntology().getOWLOntologyManager().getOWLDataFactory(), reasoner)); InferredSubClassAxiomGenerator b = new InferredSubClassAxiomGenerator(); results.addAll(b.createAxioms(reasoner.getRootOntology().getOWLOntologyManager().getOWLDataFactory(), reasoner)); return results; }

The first (the baseReasoner instance) does not produce any problem, but when asking the cloned instance to create the inferred axioms it raises a nullPointer exception:

Exception in thread "main" java.lang.NullPointerException: classExpression cannot be null at org.semanticweb.owlapi.util.OWLAPIPreconditions.checkNotNull(OWLAPIPreconditions.java:98) at uk.ac.manchester.cs.owl.owlapi.OWLDataFactoryImpl.getOWLClassAssertionAxiom(OWLDataFactoryImpl.java:1305) at uk.ac.manchester.cs.owl.owlapi.OWLDataFactoryImpl.getOWLClassAssertionAxiom(OWLDataFactoryImpl.java:1296) at org.semanticweb.owlapi.util.InferredClassAssertionAxiomGenerator.addAxioms(InferredClassAssertionAxiomGenerator.java:41) at org.semanticweb.owlapi.util.InferredClassAssertionAxiomGenerator.addAxioms(InferredClassAssertionAxiomGenerator.java:32) at org.semanticweb.owlapi.util.InferredEntityAxiomGenerator.createAxioms(InferredEntityAxiomGenerator.java:51) at OWLReasonerPersistenceTest.retrieveAllInferredAxioms(OWLReasonerPersistenceTest.java:395) at OWLReasonerPersistenceTest.main(OWLReasonerPersistenceTest.java:388)

However, when asked for the subclasses of any named class it does not raise any exception (and the results are sound and complete).

[ignazio1977]

Copy/paste error:

@Override
public boolean equals(Object obj) {
 if(obj==null) {
     return false;
 }
 if(this==obj) {
     return true;
 }
 return obj instanceof ConceptBottom;
}

This is correct for ConceptBottom, not for ConceptTop

[cbobed]

I've been testing It and the cloned reasoner seems to react to new axioms added to the TBox, but not to new axioms added to the ABox.

E.g. Asserting an object property assertion axiom does not make the reasoner infer the subject belonging to the property domain, even though having exactly the same logical axioms.

Is this the ontology listener problem or a different one?

[ignazio1977]

That's new. There is no difference between TBox and ABox as far as the manager is concerned - all axioms are treated equally. Does the original reasoner work fine in those conditions?

[cbobed]

I've dug more and I've found out that the strange behaviour happens when useIncrementalReasoning is set. However, it doesn't matter whether the instance of the reasoner is cloned or not. I attach a source code file, if you comment line 84 (setUseIncremental...), the results are coherent; if not, the baseReasoner results for OWLNothing and for Animal concepts are empty (the only difference is that I've asserted axioms just in the ABox for the baseReasoner, but both in the ABox and TBox for the cloned one).

Besides, the first time you ask for OWLNothing equivalent classes to the baseReasoner, it retrieves it correctly. After adding information just to the Abox, it does not retrieve the equivalent classes anymore. (line 229 vs 361). This behaviour happens (if you comment lines 330-336) with both instances so I think this issue should be moved to its own thread.

OWLReasonerPersistenceTest.zip

[ignazio1977]

I think that's a known bug in older versions, incremental reasoning did not work properly and should be disabled, but was incorrectly enabled by default.

[cbobed]

was it solved in JFact 1.2.4 or incremental reasoning should be avoided at all costs at this stage?

[ignazio1977]

It should be avoided, it was never completed properly.

[cbobed]

Ok :)

is there any plan to get it fixed in short-mid term? It could be really useful

[ignazio1977]

Not at present - incremental reasoning is not an easy fix, and past my abilities at the moment.

I'm at the design stage (i.e., intensely thinking about it) for another approach, which is to redesign the reasoner to use atomic decomposition - basically incorporating the work we did with Chainsaw. So instead of one monolithic kernel we'd have many small ones, to be thrown away when changes affect their atoms. This provides a result very similar to incremental reasoning, but with the potential to scale better on large, loosely connected ontologies.

However, early days is an optimistic assessment - probably best to check back in six months for anything rooted in reality.