search

microstream-one / microstream

High-Performance Java-Native-Persistence. Store and load any Java Object Graph or Subgraphs partially, Relieved of Heavy-weight JPA. Microsecond Response Time. Ultra-High Throughput. Minimum of Latencies. Create Ultra-Fast In-Memory Database Applications & Microservices.
https://microstream.one/
Eclipse Public License 2.0
560 stars 44 forks source link

StorageExceptionConsistency? #705

Open yujihoo opened 8 months ago

yujihoo commented 8 months ago

Environment Details

Describe the bug

If we have a channel count equals to 1, and the service is being accessed by more than one person to read and write into the channel_0, it shows the following error? Apart from increasing the channel count (how much should I increase as it might affect performance too?), is there any other way to solve this? I want to control the concurrency like Redis does, which makes a queue for all the incoming operations on the data instead of using multi-threads?

Error


Caused by: org.eclipse.store.storage.exceptions.StorageException: Problem in channel #0
    at org.eclipse.store.storage.types.StorageChannelTask$Abstract.checkForProblems(StorageChannelTask.java:106)
    at org.eclipse.store.storage.types.StorageChannelTask$Abstract.waitOnCompletion(StorageChannelTask.java:168)
    at org.eclipse.store.storage.types.StorageSystem$Default.startThreads(StorageSystem.java:333)
    at org.eclipse.store.storage.types.StorageSystem$Default.internalStartUp(StorageSystem.java:516)
    at org.eclipse.store.storage.types.StorageSystem$Default.start(StorageSystem.java:602)
    at org.eclipse.store.storage.types.StorageSystem$Default.start(StorageSystem.java:72)
    at org.eclipse.store.storage.embedded.types.EmbeddedStorageManager$Default.start(EmbeddedStorageManager.java:247)
    at org.eclipse.store.storage.embedded.types.EmbeddedStorageManager$Default.start(EmbeddedStorageManager.java:95)
    at org.eclipse.store.storage.embedded.types.EmbeddedStorage.createAndStartStorageManager(EmbeddedStorage.java:609)
    at org.eclipse.store.storage.embedded.types.EmbeddedStorage.start(EmbeddedStorage.java:495)
    at org.eclipse.store.storage.embedded.types.EmbeddedStorage.start(EmbeddedStorage.java:342)
    at com.nsl.dataservices.cache.eclipsestore.EclipseStoreService.getS3Manager(EclipseStoreService.java:62)
    at com.nsl.dataservices.cache.eclipsestore.EclipseStoreService.getStorageManager(EclipseStoreService.java:46)
    at com.nsl.dataservices.cache.eclipsestore.EclipseStoreService.<clinit>(EclipseStoreService.java:34)
    ... 32 more
Caused by: org.eclipse.store.storage.exceptions.StorageExceptionIoWriting
    at org.eclipse.store.storage.types.StorageFileWriter.truncateFile(StorageFileWriter.java:168)
    at org.eclipse.store.storage.types.StorageFileWriter.truncate(StorageFileWriter.java:144)
    at org.eclipse.store.storage.types.StorageFileManager$Default.handleLastFile(StorageFileManager.java:1291)
    at org.eclipse.store.storage.types.StorageFileManager$Default.initializeForExistingFiles(StorageFileManager.java:998)
    at org.eclipse.store.storage.types.StorageFileManager$Default.initializeStorage(StorageFileManager.java:873)
    at org.eclipse.store.storage.types.StorageChannel$Default.initializeStorage(StorageChannel.java:782)
    at org.eclipse.store.storage.types.StorageChannelTaskInitialize$Default.succeed(StorageChannelTaskInitialize.java:200)
    at org.eclipse.store.storage.types.StorageChannelTaskInitialize$Default.succeed(StorageChannelTaskInitialize.java:34)
    at org.eclipse.store.storage.types.StorageChannelSynchronizingTask$AbstractCompletingTask.synchronizedComplete(StorageChannelSynchronizingTask.java:78)
    at org.eclipse.store.storage.types.StorageChannelSynchronizingTask$AbstractCompletingTask.complete(StorageChannelSynchronizingTask.java:126)
    at org.eclipse.store.storage.types.StorageChannelTask$Abstract.processBy(StorageChannelTask.java:260)
    at org.eclipse.store.storage.types.StorageChannel$Default.work(StorageChannel.java:453)
    at org.eclipse.store.storage.types.StorageChannel$Default.run(StorageChannel.java:536)
    at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: org.eclipse.store.storage.exceptions.StorageExceptionConsistency: Reopened file has inconsistent size: Expected size 2830020 != actual size 3478214.
    at org.eclipse.store.storage.types.StorageLiveDataFile$Default.internalOpenWriting(StorageLiveDataFile.java:282)
    at org.eclipse.store.storage.types.StorageFile$Abstract.ensureWritable(StorageFile.java:390)
    at org.eclipse.store.storage.types.StorageFile$Abstract.truncate(StorageFile.java:366)
    at org.eclipse.store.storage.types.StorageFileWriter.truncateFile(StorageFileWriter.java:164)
    ... 13 more
hg-ms commented 8 months ago

Hello, At the first glance the exception may be caused by multiple storage instance accessing one storage target. This must not happen. Regarding concurrency and channels:

yujihoo commented 8 months ago

@hg-ms Is there any way if I want to access storage from multiple storage manager instances?

hg-ms commented 8 months ago

No, it’s not possible to access a storage target from multiple storage manager instances simultaneous except the limited Read-Only Mode.

yujihoo commented 8 months ago

Will EmbeddedStorageManager storageManager = EmbeddedStorage .Foundation() .setLockFileSetupProvider(Storage.LockFileSetupProvider()) .start(); mentioned in https://docs.eclipsestore.io/manual/storage/configuration/lock-file.html work? Facing an issue with the above code too.

hg-ms commented 8 months ago

The Lock File will help to prevent a second storage manager accessing the storage target. If a manager has locked the storage all others will throw an StorageExceptionInitialization at startup. It may happen that the lock file is not created automatically, in that case you need to create an empty named “used.lock” in the storage directory manually.