search

microstream-one / microstream

High-Performance Java-Native-Persistence. Store and load any Java Object Graph or Subgraphs partially, Relieved of Heavy-weight JPA. Microsecond Response Time. Ultra-High Throughput. Minimum of Latencies. Create Ultra-Fast In-Memory Database Applications & Microservices.
https://microstream.one/
Eclipse Public License 2.0
558 stars 44 forks source link

javafx saving observabel list resulting in "Type not persistable: "class java.util.TimerThread" #632

Open lanthale opened 10 months ago

lanthale commented 10 months ago

Environment Details

Describe the bug

I try to save the objects stored in an observable list to disk and load it again to speed up object creation. If I do with my Observable List I am getting the exception that TimerThread is not persistable. But the objects hold inside of the list are not using TimerThread.

State if the problem is easily reproducible or happens intermittently. Code that shows the problem: //init of the medialist fullMediaList = FXCollections.synchronizedObservableList(FXCollections.observableArrayList(MediaFile.extractor())); filteredMediaList = new FilteredList<>(fullMediaList, null); filteredMediaList.setPredicate(standardFilter().and(filterDeleted(showDeletedButton.isSelected()))); sortedMediaList = new SortedList<>(filteredMediaList); //code for saving to disk EmbeddedStorageManager storageManager = EmbeddedStorage.start(fullMediaList, Paths.get(Utility.getAppData())); storageManager.storeRoot();

Include stack traces or command outputs save cache... [JavaFX Application Thread] INFO one.microstream.util.logging.Logging - MicroStream Version 08.01.01-MS-GA [JavaFX Application Thread] INFO one.microstream.storage.embedded.types.EmbeddedStorageFoundation$Default - Creating embedded storage manager [JavaFX Application Thread] INFO one.microstream.persistence.types.PersistenceTypeHandlerManager$Default - Initializing type handler manager Exception in thread "JavaFX Application Thread" one.microstream.persistence.exceptions.PersistenceExceptionTypeNotPersistable: Type not persistable: "class java.util.TimerThread". at one.microstream.persistence.binary.internal.BinaryHandlerUnpersistable.guaranteeSubTypeInstanceViablity(BinaryHandlerUnpersistable.java:98) at one.microstream.persistence.types.PersistenceTypeHandlerManager$Default.lambda$recursiveEnsureTypeHandlers$0(PersistenceTypeHandlerManager.java:278) at one.microstream.persistence.binary.internal.AbstractBinaryHandlerReflective.iterateMemberTypes(AbstractBinaryHandlerReflective.java:591) at one.microstream.persistence.types.PersistenceTypeHandlerManager$Default.recursiveEnsureTypeHandlers(PersistenceTypeHandlerManager.java:271) at one.microstream.persistence.types.PersistenceTypeHandlerManager$Default.initialRegisterTypeHandlers(PersistenceTypeHandlerManager.java:714) at one.microstream.persistence.types.PersistenceTypeHandlerManager$Default.synchInternalInitialize(PersistenceTypeHandlerManager.java:984) at one.microstream.persistence.types.PersistenceTypeHandlerManager$Default.initialize(PersistenceTypeHandlerManager.java:957) at one.microstream.storage.embedded.types.EmbeddedStorageFoundation$Default.createEmbeddedStorageManager(EmbeddedStorageFoundation.java:803) at one.microstream.storage.embedded.types.EmbeddedStorage.createAndStartStorageManager(EmbeddedStorage.java:613) at one.microstream.storage.embedded.types.EmbeddedStorage.start(EmbeddedStorage.java:472) at org.photoslide.browserlighttable.LighttableController.lambda$setSelectedPath$4(LighttableController.java:329) at javafx.base@21-ea/com.sun.javafx.event.CompositeEventHandler.dispatchBubblingEvent(CompositeEventHandler.java:86) at javafx.base@21-ea/com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(EventHandlerManager.java:232) at javafx.base@21-ea/com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(EventHandlerManager.java:189) at javafx.base@21-ea/com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:58) at javafx.base@21-ea/com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114) at javafx.base@21-ea/com.sun.javafx.event.EventUtil.fireEventImpl(EventUtil.java:74) at javafx.base@21-ea/com.sun.javafx.event.EventUtil.fireEvent(EventUtil.java:54) at javafx.base@21-ea/javafx.event.Event.fireEvent(Event.java:198) at javafx.graphics@21-ea/javafx.concurrent.EventHelper.fireEvent(EventHelper.java:219) at javafx.graphics@21-ea/javafx.concurrent.Task.fireEvent(Task.java:1359) at javafx.graphics@21-ea/javafx.concurrent.Task.setState(Task.java:725) at javafx.graphics@21-ea/javafx.concurrent.Task$TaskCallable.lambda$call$1(Task.java:1437) at javafx.graphics@21-ea/com.sun.javafx.application.PlatformImpl.lambda$runLater$10(PlatformImpl.java:456) at java.base/java.security.AccessController.doPrivileged(AccessController.java:400) at javafx.graphics@21-ea/com.sun.javafx.application.PlatformImpl.lambda$runLater$11(PlatformImpl.java:455) at javafx.graphics@21-ea/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:95)

To Reproduce

Step by step instructions to reproduce the problem. Check out the master branch of github.com/lanthale/photoslide File: LighttableController.java:294 for init File: LighttableController.java:329 for saving to disk File: MediaFile (object to save to disk)

compile/test

compile via maven: mvn clean install run: mvn clean javafx:run Add one location with images select one of the directories with images and the exception will occure.

Expected behavior

A clear and concise description of what you expected to happen. My expections are the the object fullMediaList is saved to disk

Screenshots

If applicable, add screenshots to help explain your problem.

Additional context

Add any other context about the problem here.

hg-ms commented 10 months ago

You may have a look in the PersistenceTypeDictionary.ptd file in the storage directory after the exception. It is a simple human readable text file that lists all classes and their fields that are in the object graph. This allows you to identify the problematic classes that reference java.util.TimerThread. Maybe you can use the transient keyword to exclude the problematic sub-graph from the persistence.

lanthale commented 10 months ago

I have tried that but removing the MediaPlayer does not help. Neither the transient keyword nor removing the MediaPlayer completely from the code is removing that issue. Can I block somehow a deep copy or is there another way you can advise ?

hg-ms commented 10 months ago

Did you try to delete the storage that threw the exception after removing the MediaPlayer? When an incompatible type is encountered the first time it is added to the type-dictionary even if it can’t be persisted. At the next startup you’ll get the PersistenceExceptionTypeNotPersistable again when the storage initializes even if the problematic types are no more part of the graph. Alternatively you can try to edit the type-dictionary and just delete the problematic entries.

lanthale commented 10 months ago

Now I have deleted the file and passed over this issue heading into the next one:

ERROR one.microstream.storage.embedded.types.EmbeddedStorageManager$Default - Exception occurred while initializing embedded storage manager one.microstream.persistence.exceptions.PersistenceException: Lambdas are not supported as they cannot be resolved during loading due to insufficient reflection mechanisms provided by the (current) JVM.

JDK is: openjdk version "20" 2023-03-21 OpenJDK Runtime Environment Temurin-20+36 (build 20+36) OpenJDK 64-Bit Server VM Temurin-20+36 (build 20+36, mixed mode)

Any tip on that error message ?

hg-ms commented 10 months ago

The error seems to be caused by a type that assigns a lambda function to a non-transient field. From your example code I assume that it is Javafx.collections.FXCollections.SynchronizedObservableList. This class assigns a lambda to the listener field. Unfortunately, lambdas can’t be persisted by Microstream. To work around that I’d suggest using the standard java collection instead of the FXCollections if possible.

lanthale commented 10 months ago

I have now moved to an arraylist but still getting an error message. Is there a way which property cannot be stored ?

PersistenceExceptionTypeNotPersistable: Type not persistable: "class com.sun.javafx.collections.ObservableListWrapper$1$1". Details: Synthetic classes ($1 etc.) are not reliably persistable since a simple reordering of source code elements would change the name identity of a class. For a type system that has to rely upon resolving types by their identifying name, this would silently cause a potentially fatal error. If handling synthetic classes (e.g. anonymous inner classes) is absolutely necessary, a custom one.microstream.persistence.types.PersistenceTypeResolver can be used to remove the exception and assume complete responsibility for correctly handling synthetic class names.

lanthale commented 10 months ago

I have found out that every property class from javafx is not working.

As soon as I add only e.g. SimpleBooleanProperty class then I get the error message PersistenceExceptionTypeNotPersistable: Type not persistable: "class com.sun.javafx.collections.ObservableListWrapper$1$1". Details: Synthetic classes ($1 etc.) are not reliably persistable since a simple reordering of source code elements would change the name identity of a class. For a type system that has to rely upon resolving types by their identifying name, this would silently cause a potentially fatal error. If handling synthetic classes (e.g. anonymous inner classes) is absolutely necessary, a custom one.microstream.persistence.types.PersistenceTypeResolver can be used to remove the exception and assume complete responsibility for correctly handling synthetic class names.

Any idea howto circumstance that issue ?

hg-ms commented 10 months ago

It seems that most JavaFX classes are not suitable for serialization. There is no easy way to get Microstream serializing those classes. In case of the synthetic classes and fields we can’t guarantee that their names are constant a cross different builds as they are generated by the compiler therefore, they can’t be persisted.

As there is no easy way to work around that, I’d suggest not persisting the JavaFX types directly. Instead you may create an independent data model holding the required data.