Request for Comments: Is the historical context useful to you? Otherwise the chainweb-data dependency can be dropped.

[Takadenoshi]

One of the requirements we set for this project was the ability to provide some historical data for the queries we serve. When you query for an account or a module, you will receive as many historical entries as the ?limit= query parameter you specify*.

This is the reason chainweb-data is a dependency. Since we need that "path" to read historical data, we also use it for all data reads instead of chainweb-node. If builders/users have their own historical data sources or do not need this feature, then removing the chainweb-data dependency and switching to reading from chainweb-node will simplify both the chainweb-stream implementation and its deployment / maintenance.

* The exception is a bug I just found regarding the very first "instance" of query you make: if cw-stream does not have that account/module/event data cached, you will receive the past 100 transactions regardless of ?limit. I will look into this.

[Takadenoshi]

For quick feedback:

Give this comment a 👍 to KEEP the ability to provide historical context and the chainweb-data dependency

Give this comment a 👎 to REMOVE the ability to provide historical context and the chainweb-data dependency

[trendzetter]

the historical context can be had directly from chainweb-data without the need for streaming I think. imo streaming is about real-time events data. Curious to hear other opinions

[daplcor]

I was going to say the same thing as George. We have historical(whether data-node or other index db solutions), meaning initial data is seeded at loading. This is only impacting current operations, which should catch real-time streams. For more dApps, I suspect you will find agreement and it's not because the technology stack is inadequate or ineffective. We want simple, easy, and space starts to come at a premium as the chains grow. You are on the right track, and you'll make a friend for life if we can cut out the data node for streaming. If there is anything I can do to help or test, don't hesitate since we all want the best possible experience for the chain as we grow.

[Jad-KMC]

Please cut out the need for chainweb-data, the streaming events are valuable on their own