Point72 / csp

csp is a high performance reactive stream processing library, written in C++ and Python
https://github.com/Point72/csp/wiki
Apache License 2.0
191 stars 33 forks source link

Graphs created as part of a `CspPerspectiveTable` do not memoize #263

Open AdamGlustein opened 4 months ago

AdamGlustein commented 4 months ago

Describe the bug

When you create a graph as part of a CspPerspectiveTable it does not memoize nodes.

To Reproduce

Minimal repro

import csp
from datetime import datetime, timedelta
from PIL import Image

import perspective
from csp.impl.pandas_ext_type import TsDtype
from csp.impl.pandas_perspective import CspPerspectiveTable
import pandas as pd

df = pd.DataFrame()
df.insert(0, 'one1', pd.Series(csp.const(1.0), dtype=TsDtype(float)), True)
df.insert(1, 'one2', pd.Series(csp.const(1.0), dtype=TsDtype(float)), True)

table = CspPerspectiveTable(df, index_col='timestamp', keep_history=False, localize=True)
g = table.graph

# Note I use a roundabout to call show_graph since I'm in Jupyterlab
# You can replace the code below with a simple show_graph call
buffer = csp.showgraph.generate_graph(g)
image = Image.open(buffer)
display(image)

This gives an image clearly showing the const adapter is not memoized:

image

However, if I "manually" memoize the const nodes by only creating one and providing it both index 0 and 1 of the CspPerspectiveTable like:

df = pd.DataFrame()
u = csp.const(1.0)
df.insert(0, 'one1', pd.Series(u, dtype=TsDtype(float)), True)
df.insert(1, 'one2', pd.Series(u, dtype=TsDtype(float)), True)

We get what we expect:

image

Expected behavior

Memoization should work like it does for any other graph

Error Message

Runtime Environment

0.0.3
3.10.12 | packaged by conda-forge | (main, Jun 23 2023, 22:40:32) [GCC 12.3.0]
linux

Additional context