In a large space with lots of elements, element rendering resolution becomes a bottle neck and a subject of compromise. If the element resolution is low, images and text look blocky and rough. On the other hand, if the resolution is high for all elements regardless of their apparent size, the memory consumption grows high. An image of 2000x2000 pixels consumes about 16 MB of GPU memory. For 100 elements that implies 1600 MB.
Tile stacks and quadtrees are one way to solve the problem and used in many geographical applications. There every z level the image is split in four subimages and the number of images kept low by removing too large and too small images.
In Tapspace, the content is more heterogeneous than just square tiles. Therefore we need to generalise the loading algorithm of tile stacks. The stack can be infinite. Therefore the notion of fractals is the best to describe the concept. Also, we can see that a certain content can have a various levels of representation abstraction. We call these abstraction stages where some stages are more abstract or more reduced and some are more concrete or more granular.
We can distinguish a life cycle of an element in a fractal:
the element is created
the element is added to DOM
the element is made visible and becomes part of the DOM layout
the element is hidden
the element is removed from DOM
the element is erased by a garbage collector
These life cycle stages can be separate or simultaneous. The element can be immediately created and shown in DOM as well as removed all at once. Should the fractal API allow control of every stage?
A general-purpose fractal loader would provide control when these stage steps happen. The main driver for triggering the stages is the orthogonal distance (along z) from viewport image plane to the element, because that directly determines the apparent resolution. In the optimal case, all elements are rendered in the resolution that matches the viewport. In a good compromise, because the rendering and laying out the element contents is a computationally intensive task, the elements are kept close to viewport resolution but refreshed only when truly needed. The viewport idle event can be a good moment for the refresh.
The abstraction stages need to have some temporal overlap to prevent flicker. In the case of tiles, the large tile can be removed only after all its subtiles have been downloaded and rendered. Therefore each stage step requires a warm-up period, a kind of a staging phase, and some logic when the more abstract representation can be removed.
Triggering the abstraction steps also needs some temporal or spatial threshold. A shaky hand or jittery pointer can cause constant load and reload when accidentally executed at the stage range limit, possibly even 60 times a second. This can be avoided with hysteresis by either placing a temporal or spatial distance that the viewport needs to wait or travel before another abstraction step can be made.
There is no single rule when to hide or remove the element of an abstract stage. With tiles, the subtiles need to be loaded first. With networks and tree structures, the rendering of child nodes should not cause the parent node to escape, instead the parent will be visible until it is too large or too far away from the viewport. Also, the node itself can have multiple representations and the child nodes may appear only in one of them.
The biggest problem is how to model all these features to a usable API. How the app developer should define the abstraction stages without running into complex spaghetti code but also without being limited to certain type of stage loading such as the tile stack. There are various options:
measuring events: the app dev listens to changes in element distanced and implements all the loading logic ad-hoc.
stage descriptions: the app dev writes a constructor and a destructor for each abstraction stage and then associates them with depth ranges. Tapspace handles the logic that calls the methods.
fractal template: the app dev writes a constructor for a subspace in the fractal. The constructor acts like a template for all the stages. The constructor receives data like its depth in the network, navigation direction, and other accumulated data, which allow the template to vary its content and layout. Tapspace is responsible for rendering the template, positioning it in the space, and destructing it when needed. In this option, individual components have only their initial representation, although the template could be modelled for both.
The fractal template approach feels most practical. The challenge is how to apply it to cover all the possibilities above. Especially the dependent destruction of tile loading requires some special treatment, as the child templates must be able to signal the parent that their rendering is finished, and the parent must be able to expect that certain children may signal the parent to self-destruct. This introduces the difference between dependent and independent children. Where former provide more granular version of the same content, and the latter add non-duplicate content that allow or even expect the parent to exist.
In a large space with lots of elements, element rendering resolution becomes a bottle neck and a subject of compromise. If the element resolution is low, images and text look blocky and rough. On the other hand, if the resolution is high for all elements regardless of their apparent size, the memory consumption grows high. An image of 2000x2000 pixels consumes about 16 MB of GPU memory. For 100 elements that implies 1600 MB.
Tile stacks and quadtrees are one way to solve the problem and used in many geographical applications. There every z level the image is split in four subimages and the number of images kept low by removing too large and too small images.
In Tapspace, the content is more heterogeneous than just square tiles. Therefore we need to generalise the loading algorithm of tile stacks. The stack can be infinite. Therefore the notion of fractals is the best to describe the concept. Also, we can see that a certain content can have a various levels of representation abstraction. We call these abstraction stages where some stages are more abstract or more reduced and some are more concrete or more granular.
We can distinguish a life cycle of an element in a fractal:
These life cycle stages can be separate or simultaneous. The element can be immediately created and shown in DOM as well as removed all at once. Should the fractal API allow control of every stage?
A general-purpose fractal loader would provide control when these stage steps happen. The main driver for triggering the stages is the orthogonal distance (along z) from viewport image plane to the element, because that directly determines the apparent resolution. In the optimal case, all elements are rendered in the resolution that matches the viewport. In a good compromise, because the rendering and laying out the element contents is a computationally intensive task, the elements are kept close to viewport resolution but refreshed only when truly needed. The viewport idle event can be a good moment for the refresh.
The abstraction stages need to have some temporal overlap to prevent flicker. In the case of tiles, the large tile can be removed only after all its subtiles have been downloaded and rendered. Therefore each stage step requires a warm-up period, a kind of a staging phase, and some logic when the more abstract representation can be removed.
Triggering the abstraction steps also needs some temporal or spatial threshold. A shaky hand or jittery pointer can cause constant load and reload when accidentally executed at the stage range limit, possibly even 60 times a second. This can be avoided with hysteresis by either placing a temporal or spatial distance that the viewport needs to wait or travel before another abstraction step can be made.
There is no single rule when to hide or remove the element of an abstract stage. With tiles, the subtiles need to be loaded first. With networks and tree structures, the rendering of child nodes should not cause the parent node to escape, instead the parent will be visible until it is too large or too far away from the viewport. Also, the node itself can have multiple representations and the child nodes may appear only in one of them.
The biggest problem is how to model all these features to a usable API. How the app developer should define the abstraction stages without running into complex spaghetti code but also without being limited to certain type of stage loading such as the tile stack. There are various options:
The fractal template approach feels most practical. The challenge is how to apply it to cover all the possibilities above. Especially the dependent destruction of tile loading requires some special treatment, as the child templates must be able to signal the parent that their rendering is finished, and the parent must be able to expect that certain children may signal the parent to self-destruct. This introduces the difference between dependent and independent children. Where former provide more granular version of the same content, and the latter add non-duplicate content that allow or even expect the parent to exist.