microsoft / FASTER

Fast persistent recoverable log and key-value store + cache, in C# and C++.
https://aka.ms/FASTER
MIT License
6.32k stars 567 forks source link

memory rising so much #796

Closed darkLightx closed 1 year ago

darkLightx commented 1 year ago
 _fasterKv = new FasterKV<TKey, TValue>(

                size: 262144,
                logSettings: new LogSettings
                {
                    LogDevice = Devices.CreateLogDevice( @"C:\tmp\test.log",
                    preallocateFile: true,
                    deleteOnClose: true),
                    ObjectLogDevice = Devices.CreateLogDevice(@"C:\tmp\test.obj.log",
                    preallocateFile: true,
                    deleteOnClose: true),
                    PageSizeBits = 16,
                    MemorySizeBits = 22,
                    ReadCacheSettings = new ReadCacheSettings
                    {
                        MemorySizeBits = 22,
                        PageSizeBits = 16,
                    }
                },
                serializerSettings: serializer);

My configuration is as above. When I write 194338 data, the memory occupies 594MB; When I write 3886759 data, the memory occupies 4200MB; Why is the memory rising so much? Why can't we effectively control the memory occupation.

badrishc commented 1 year ago

You can take a look at the docs here for an explanation of memory usage and tuning: https://microsoft.github.io/FASTER/docs/fasterkv-tuning/#log-memory-size-with-c-heap-objects