cortoproject / corto

A hierarchical object store for connecting realtime machine data with web applications, historians & more
https://www.corto.io
MIT License
86 stars 14 forks source link

Add support for lazy evaluation of references #675

Open SanderMertens opened 6 years ago

SanderMertens commented 6 years ago

Currently, when an object is being resumed into the in-memory store, its references are immediately resumed as well, typically as a result of serializers performing a corto_lookup or a corto_resolve. While this is convenient in some cases (a developer can walk easily walk references) the behavior is "viral", in that it can easily resume a large part of the complete object graph into the store.

To prevent this from happening, the typesystem and serializers should allow for "lazy" references, that are only resumed when the user explicitly asks for it.

Ideally, the difference between lazy and non-lazy references is hidden from the user, and is encapsulated in corto_set_ref (already exists) and corto_get_ref (doesn't exist yet) functions.

SanderMertens commented 5 years ago

To support lazy references for members, collection elements, function parameters and function return values, a single mechanism is required to indicate whether a reference is lazy or not. Whereas this feature could be easily added for members by adding a new "lazy" constant to corto/lang/modifierMask, this would not make it available for elements and functions.

To address this, a new feature should be added to the typesystem that "wraps" an existing type, and modifiers its behavior (making it a lazy reference). For example:

class Foo {
    ...
}

struct Bar {
    m: lazy_ref[Foo] // wraps Foo, turns it into a "lazy type"
}

This capability would also be useful in a couple of other areas, like creating reference members (or elements, ...) to non-reference types:

struct Foo {
    ...
}

struct Bar {
    m: ref[Foo] // Create a reference member to non-ref type "Foo"
}

Yet another capability of such a design would be to create typedefs:

typedef Foo: Point

struct Bar {
    m: Foo
}

To support this, a new type kind (TYPEDEF) must be introduced that allows one type to wrap another type.