ShiftLeftSecurity / overflowdb

ShiftLeft OverflowDB
Apache License 2.0
112 stars 21 forks source link

Memory optimisation: deduplicate identical `AdjacentNodes.offsets` arrays #334

Open mpollmeier opened 1 year ago

mpollmeier commented 1 year ago

In large graphs we're wasting ~8% of the heap for duplicate arrays in AdjacentNodes.offsets. That array is only ever mutated in AdjacentNodes.setOffset in a very constraint and straighforward way, so we could easily optimise it.

E.g. we could have a global TIntObjectMap as a cache that maps from the checksum of a given offset array to the array.