search

kmgreen2 / agglo

Agglo: A Process-Anywhere Framework for Event Stream Processing
https://kmgreen2.github.io/agglo/
MIT License
2 stars 0 forks source link

Make Entwine Process #6

Closed kmgreen2 closed 3 years ago

kmgreen2 commented 3 years ago

All of the entwine code is more-or-less independent of the core agglo stuff. Make it part of it!

kmgreen2 commented 3 years ago

This will require some additional glue on top of the existing implementation in the entwine package.

  1. The ability to setup an accessible ticker. That is, create a process that utilizes a cloud KVStore to tick (via Lambda) and another process that allows a caller to Anchor() (also via a lambda)
  2. Create a process that will append a payload a substream
  3. The backend will do Append() via Spawning a the process in #2
kmgreen2 commented 3 years ago

Quick update:

StreamStore and TickerStore will both be Tee types. A StreamStore tee will conditionally Append(), while a TickerStore tee will Anchor()

kmgreen2 commented 3 years ago

Another update:

Let's just make Entwine a process. The process will currently do the following:

  1. Upload the serialized map to a chosen ObjectStore
  2. Create an immutable message based on the passed-in map
  3. Append the immutable message to a specific substream
  4. Conditionally Anchor the substream with a provided ticker stream

In the future, we can also add support to not do step 1, but get the object descriptor from the input map.

kmgreen2 commented 3 years ago

This is mostly done, just need to add to the config and React components.

kmgreen2 commented 3 years ago

Creating new issue to track the config/React portion