pcdshub / superscore

Configuration Management for EPICS PVs
Other
1 stars 2 forks source link

Implement Lazy Loading #21

Open tangkong opened 2 months ago

tangkong commented 2 months ago

Description

Superscore should support lazy loading of Entry's, in order to improve performance and memory load.

Context

Consider the case where we want to present a preview of the results of a filter operation. This filter could match Entry's with significant nesting. If there were no lazy loading we would have to grab all the children information at once, and much of that information would be useless to the user.

A more extreme case is the a tree-view of the entire database. Without some form of lazy loading this would require we grab the entire database contents with each app instance.

Some discussion had in #5

Implementation thoughts

One option is to allow UUID references wherever there is a reference to another data object. If all Entry's are fully UUID-ified, then grabbing any Entry stops at depth 1. We could then request further information as needed. The in-memory Backend could hold an Entry-cache, mapping previously-queried Entry's to their UUIDs. Text-based deserialization shouldn't be too much of a problem here, provided all our dataclasses have distinct signatures.

These are my thoughts, at least

shilorigins commented 2 months ago

Direct link to the relevant thread