search

dm4ml / motion

Framework for building and maintaining self-updating prompts for LLMs
https://dm4ml.github.io/motion/
58 stars 4 forks source link

Thinking about orchestrating / serving an application of components #259

Closed shreyashankar closed 9 months ago

shreyashankar commented 10 months ago

Suppose we have a python module with a bunch of components, and we want to serve this.

Basically we'd want to serve a rest api such that components can easily be accessed. So a user could define a deployment script that:

  1. Looks for a requirements.txt in the same directory
  2. Imports all motion components they want to deploy
  3. Bundles these components into an "app" & returns an app id & authentication token
  4. Serves basically a fastapi app that listens for any request to the app id & dataflow key & props, and runs the dataflow accordingly. It should return the result to the user and

The client library should connect to a component instance via app id and auth token.

shreyashankar commented 10 months ago

Maybe we can keep components cached in the server somehow, in case there are successive calls

shreyashankar commented 9 months ago

I don't think we need a Python client library -- since they are created on the fly in the server.

We could build a JS / TS sdk