Currently, objects/arrays must be constructed using ModelPool::addObject or ModelPool::addArray with a previously constructed MemberRange. So it is necessary to know the object or array members ahead of time, which introduces unnecessary complexity, and a temporary vector allocation into model construction code.
To allow for objects/arrays with dynamic size, an allocator view must be implemented on top of the member column vector, which manages growable ranges within it. We propose to call this view a VectorArena.
The VectorArena is wrapper around the segmented_vector. It keeps track of embedded forward-linked spans of size 1,2,..2^n. When an array grows beyond the capacity c of its current last span, a new span of size c*2 is allocated and becomes the new last span. This is then set as the next member of the previous last span, and as the last member of the first span in the chain. Usually append will be lock-free, only growth needs a lock.
The VectorArena tracks allocated spans in two collections:
The heads collection is the list of root spans, which are the heads of allocated span chains. Other ModelNodes reference vectors as indices in this list. The index of a span object in this list replaces the current MemberRange type.
The continuations collection is the list of spans which are allocated when a previous span is exhausted.
Note: Serialization and deserialisation will compact and defragment the VectorArena. After a seralisation roundtrip, continuations are always empty, and the capacity of each head span equals its size.
Currently, objects/arrays must be constructed using
ModelPool::addObjectorModelPool::addArraywith a previously constructedMemberRange. So it is necessary to know the object or array members ahead of time, which introduces unnecessary complexity, and a temporary vector allocation into model construction code.To allow for objects/arrays with dynamic size, an allocator view must be implemented on top of the member column vector, which manages growable ranges within it. We propose to call this view a
VectorArena.The
VectorArenais wrapper around the segmented_vector. It keeps track of embedded forward-linked spans of size 1,2,..2^n. When an array grows beyond the capacitycof its current last span, a new span of sizec*2is allocated and becomes the new last span. This is then set as thenextmember of the previous last span, and as thelastmember of the first span in the chain. Usually append will be lock-free, only growth needs a lock.The
VectorArenatracks allocated spans in two collections:headscollection is the list of root spans, which are the heads of allocated span chains. Other ModelNodes reference vectors as indices in this list. The index of a span object in this list replaces the currentMemberRangetype.continuationscollection is the list of spans which are allocated when a previous span is exhausted.Note: Serialization and deserialisation will compact and defragment the VectorArena. After a seralisation roundtrip,
continuationsare always empty, and the capacity of each head span equals its size.