search

flyteorg / flyte

Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
https://flyte.org
Apache License 2.0
5.81k stars 660 forks source link

[Core feature] Realtime applications on Flyte #3247

Open davidmirror-ops opened 1 year ago

davidmirror-ops commented 1 year ago

Motivation: Why do you think this is important?

Acccording to @dylanwilder:

"we have an use case where the overhead of spinning up flyte components provides some lag in our user experience, but we can't duplicate business logic into a web backend"

Goal: What should the final outcome look like, ideally?

"we have user interactive loops for which we currently wait for long processing calculations ("what if" scenario modeling). some of these models take a while to run and a long feedback loop is expected. others are quick (think mathematical formulas) and expectation is realtime feedback on parameter changes. so not the typical streaming pipeline application but we could definitely leverage it. the major bottleneck in some tasks is spool times, and in others we have use cases where we could run caculations for smaller sets of data for faster output but spool time would quickly become the bottleneck here as well"

Describe alternatives you've considered

TBD

Propose: Link/Inline OR Additional context

No response

Are you sure this issue hasn't been raised already?

Have you read the Code of Conduct?

kumare3 commented 1 year ago

@davidmirror-ops I have a few ideas and would love to propose an RFC. But first can be dive a little deeper. What would be expectations. Realtime is a broad spectrum, do we want to maintain similar levels of "state" guarantees, fully reproducilble, exactly one, ability to recover from anywhere, partial caching etc? or is it ok to have a lower durability with extremely fast runtimes - in order of milliseconds. cc @dylanwilder

dylanwilder commented 1 year ago

fwiw, for our use case we don't need anything super fancy. we just would like to avoid k8s spool time, eg by having docker containers pre-warmed and possible circumventing heavier IO by exposing an interface a web service could hit.

I admit this is not truly the "streaming" use case - nor is it a very standard one. Though it's possible we could make streaming functionality work for us.

kumare3 commented 11 months ago

@dylanwilder we are indeed working on this at Union, would you be interested in talking