search

RBMHTechnology / eventuate

Global-scale event sourcing and event collaboration with causal consistency (This project is in maintenance mode. Only critical bugs will be fixed, but there is no more feature development.).
http://rbmhtechnology.github.io/eventuate/
Apache License 2.0
709 stars 99 forks source link

Memory consumption of `LeveldbNumericIdentifierStore` #402

Open magro opened 6 years ago

magro commented 6 years ago

Currently we're getting OutOfMemoryErrors in a node that's just replicating events, running with leveldb. Because the node is only replicating events we assumed that it does not need much memory, therefore it's running with 1G max heap.

The heap dump analysis showed that eventuate's LeveldbNumericIdentifierStore consumes ~1.3GB RAM, which is allocated by the internal idMap (with ~6.5M entries): sshot_sync001_heapdump_idstore

AFAICS the idMap stores aggregate ids, right?

For now we'll increase the max heap size of the java process (which we could also keep as solution, assuming that the string ids are aggregate ids). It might still be considered to reduce the memory requirements of the leveldb event log.

Because the LeveldbNumericIdentifierStore stores Int -> String (numeric id to string id) in leveldb, it's currently not possible to resolve a string id to its numeric id via leveldb (in def numericId(id: String): Int). I.e. we cannot simply get rid of the idMap. If we don't want to also store the String -> Int mapping in leveldb we'd have to use a more memory efficient Map implementation for this use case.

WDYT?

krasserm commented 6 years ago

We should consider the Int->String storage design as obsolete (it has historical reasons which do not apply any more) and directly store the String->Int mapping in LevelDB (as we don't need a key ordering here and hence don't need keys of equal length). For backward compatibility this requires DB migration.

magro commented 6 years ago

Ok, thanks for the hint!