search

deephaven / deephaven-core

Deephaven Community Core
Other
252 stars 80 forks source link

updateView should take advantage of SharedContext for chained formulas #5230

Open rbasralian opened 6 months ago

rbasralian commented 6 months ago

A common pattern is:

.update("Parsed=foo()").updateView("A=Parsed.getA()", "B=Parsed.getB()")

So that foo() is not called many times. For large tables, especially a source blink table we don’t like actually taking up all that memory permanently.

.updateView("Parsed=foo()", "A=Parsed.getA()", "B=Parsed.getB()")

When the reader for column A reads “A” within a shared context, it should remember A’s chunk and allow it to be used for B.

This is particularly relevant for the UI, for table loggers, and for downstream aggregations such as .view("A", "B").sumBy().

rcaudy commented 6 months ago

I think the description is thinking about this wrong/too-small. We could make getChunk caching be a reality across the board. That is, a GetContext could use a shared context "magic bag" to hold its result, keyed by the ChunkSource.

rcaudy commented 6 months ago

The plan is to address this after some refactorings coming in https://github.com/deephaven/deephaven-core/pull/5194.

devinrsmith commented 2 months ago

This is also relevant for a new table operation based on ObjectProcessor we are considering.