search

CoreyKaylor / Lightning.NET

.NET library for LMDB key-value store
Other
399 stars 82 forks source link

LightningEnvironment.Dispose is very slow (GC issue?) #79

Closed slav closed 7 years ago

slav commented 7 years ago

I'm trying to benchmark LMDB .NET to see if it fits our scenario. Right now I'm just dumping 100k values into it as fast as I can with batches 1000 at a time. Everything works great until I get to disposing the environment.

When I do LightningEnvironment.Dispose the application hangs for about 30 minutes with high (50%) cpu usage and occasional access to the disk.

I'm using default settings, so I assume all commits were made to the disc and in theory I can just close the entire app without clean up and all should be good.

While dispose is hanging it looks like it's doing a lot of GCing. According to the performance counters, about 50% of the time is spent in GC.

Any idea what might be causing this and how to fix this?

slav commented 7 years ago

Btw, on side note. I suspect issue might be in _pinnedConfig.Dispose() but have no idea.

Also, you're doing mdb_dbi_close, but it's recommended not to use it under normal circumstances: http://104.237.133.194/doc/group__internal.html#ga52dd98d0c542378370cd6b712ff961b5

slav commented 7 years ago

Fixed the issue. Didn't correctly dispose all objects on every transaction and so afterwards env was trying to dispose all object that were still open.