DB file format changes

[tzaeschke]

This is a compound issue to collect planned/required file format changes.

Root Pages:

• [ ] Move commit counter into protected block
• [ ] Page version info
• [ ] root page counter (leave at 2 for now)

Index management:

• [ ] Specify Index class in stored index to support different index implementations (and versions)
• [ ] Support for different String-magic-number algorithms (implement via different index class?)
• [ ] Support for multi-field indexes
• [ ] Support for aggregate indexes across objects (path index) or classes (really?)

Index capabilities:

• [ ] Use prefix sharing index as standard
• [ ] Store min/max/cnt values for each subtree -> provide aggregate methods (min/max/avg/cnt/...)

Page storage:

• [ ] Store page offset of objects in page and store only page-id in OID index
• [ ] Page overflow: stop splitting int/long/double/...! We should only split arrays/string. This avoids allocating ByteBuffers and arrays.
• [ ] Dow we really need to align multi-byte entries to their multiples? May be useful for arrays (faster access). But for primitives?

Object storage:

• [ ] Store characters/string with 2 bytes per char (currently one byte ASCII). Also, make this configurable. Actually, Java 9 automatically stores Strings internally as 1-byte, unless 2-bytes are required. Can we do the same?
• [ ] Allow storing enums as String i.o. 'int'
• [ ] Do not store Schema-OID of target object
• [ ] Fix NULL storage, see Data(De)Serializer
• [ ] Replace standard collections with ZooDB collections: Avoid materializing objects when materializing a collection
• [ ] Collection storage format: Allow queries (map.get(x) == y) etc on serverside without materializing the collection. Store collections such that this is possible.

Schema:

• [ ] Support for 'named' fields (via @Column etc).
• [ ] Support for @Embedded, may require reference to SCO/Embedded class
• [ ] Support for initialization value for schema evolution
• [ ] Support for initialization functions (at least preliminary, provide simple Object-ref?)
• [ ] Support for Field storage options, such as String (ASCII, UTF-8, UTF-16, ...), enum (int, String, ...), ...