When upsert ArgumentOutOfRangeException

[cschuchardt88]

When I keep Upsert over and over again i get the following error System.ArgumentOutOfRangeException: 'Non-negative number required. (Parameter 'count')' It happens at random. Now i am calling a session on multiple threads for reading.

Call Stack

System.Private.CoreLib.dll!System.IO.BinaryReader.ReadBytes(int count) Line 476
    at /_/src/libraries/System.Private.CoreLib/src/System/IO/BinaryReader.cs(476)
FASTER.core!FASTER.core.ByteArrayBinaryObjectSerializer.Deserialize(out byte[] obj) Line 29
    at D:\a\1\s\cs\src\core\Index\Interfaces\ObjectSerializer.cs(29)
FASTER.core!FASTER.core.GenericAllocator<byte[], byte[]>.Deserialize(byte* raw, long ptr, long untilptr, FASTER.core.Record<byte[], byte[]>[] src, System.IO.Stream stream) Line 885
    at D:\a\1\s\cs\src\core\Allocator\GenericAllocator.cs(885)
FASTER.core!FASTER.core.GenericAllocator<byte[], byte[]>.AsyncReadPageWithObjectsCallback<FASTER.core.Empty>(uint errorCode, uint numBytes, object context) Line 661
    at D:\a\1\s\cs\src\core\Allocator\GenericAllocator.cs(661)
FASTER.core!FASTER.core.LocalStorageDevice._callback(uint errorCode, uint numBytes, System.Threading.NativeOverlapped* pOVERLAP) Line 78
    at D:\a\1\s\cs\src\core\Device\LocalStorageDevice.cs(78)
System.Private.CoreLib.dll!System.Threading.ThreadPoolTypedWorkItemQueue<System.Threading.PortableThreadPool.IOCompletionPoller.Event, System.Threading.PortableThreadPool.IOCompletionPoller.Callback>.System.Threading.IThreadPoolWorkItem.Execute() Line 1143
    at /_/src/libraries/System.Private.CoreLib/src/System/Threading/ThreadPoolWorkQueue.cs(1143)
System.Private.CoreLib.dll!System.Threading.ThreadPoolWorkQueue.Dispatch() Line 924
    at /_/src/libraries/System.Private.CoreLib/src/System/Threading/ThreadPoolWorkQueue.cs(924)
System.Private.CoreLib.dll!System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart() Line 77
    at /_/src/libraries/System.Private.CoreLib/src/System/Threading/PortableThreadPool.WorkerThread.cs(77)

System.ArgumentOutOfRangeException
Non-negative number required. (Parameter 'count')
   at System.IO.BinaryReader.ReadBytes(Int32 count)
   at FASTER.core.ByteArrayBinaryObjectSerializer.Deserialize(Byte[]& obj)
   at FASTER.core.GenericAllocator`2.Deserialize(Byte* raw, Int64 ptr, Int64 untilptr, Record`2[] src, Stream stream)
   at FASTER.core.GenericAllocator`2.AsyncReadPageWithObjectsCallback[TContext](UInt32 errorCode, UInt32 numBytes, Object context)
   at FASTER.core.LocalStorageDevice._callback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)
   at System.Threading.ThreadPoolTypedWorkItemQueue`2.System.Threading.IThreadPoolWorkItem.Execute()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()

[badrishc]

If you can provide a stand alone repro, we can look into it.

[cschuchardt88]

Problem with reproducing the error is the application is a plugin for a blockchain that uses faster as storage, that i'm making.

Sessions are not multi-thread safe? I think that is the cause of the error, plus the hanging of the session. What I have is two different threads on the same session. One thread importing blockchain data (just upserting) and the other thread reading data. At some point the session im guessing is locking up and sometimes i get that error like above comment. Also the import is slow compared to leveldb or rocksdb. But i think it has to do with my seek function. Maybe you can point me in the right direction for getting optimal speed and disk persistence. It's very hard for someone new starting out. I read all the samples, but don't really understand what faster is doing. Plus code commits and documentation is are very generic.

Here is the code i found to work without the error. Each method can be called on another thread at any giving time. So i created a session at method level instead of using the session at class level, that fixed the error and stopped the hanging of the session. Maybe you point me in the right start direction for optimal speed and disk persistence. thank you https://github.com/cschuchardt88/neo-modules/blob/FasterDBStore/src/FasterDBStore/FasterStore.cs

[badrishc]

Sessions are mono-threaded. You can use a pool of sessions to avoid two threads using the same session.

Example: https://github.com/microsoft/FASTER/blob/main/cs/playground/AsyncStress/SerializedFasterWrapper.cs

[cschuchardt88]

thank you that works. But still slow only 9-10 blocks per second compared to leveldb which is 50-100 blocks a sec. Has to do with seeking through the whole database then ordering them. Faster needs some kind of seek function for seeking forward and backwards on keys. thanks again for the help.

[badrishc]

Correct, FASTER does not include a range index. It is designed for point reads, upserts, read-modify-writes, and full scans.

[cschuchardt88]

is there anything that i can hook into; so i can order the keys? Or what do you recommend i do for seeking/ordering keys in faster?