breadboard-ai / breadboard

A library for prototyping generative AI applications.
Apache License 2.0
95 stars 19 forks source link

Implement a simple ELT provider #1473

Open dglazkov opened 2 months ago

dglazkov commented 2 months ago

As mentioned in #1037, Breadboard wants to have an extract/load/transform (ELT) provider that allows managing things other than text: PDFs, docs, images, videos, audio.

Here's a couple of scenarios:

Let's figure out how to build the simplest-possible ELT provider that relies on IDB. Here's what we need:

dglazkov commented 2 months ago

Let me ponder the multimodal content in config first

dglazkov commented 2 months ago

There's something here about saving an llm-content as one item. For instance, imagine I have a board that represents a single turn of a chat. The board takes llm-content as input and llm-content as output (aka "worker shape"). It would be exceedingly cool if I could save the output and then, when running the board again, pick the saved llm-content from some store of saved things. This would allow me to debug worker-shaped boards very easily and even simulate multiple turns

dglazkov commented 2 months ago

omg, I just realized something. With FSAPI, we can save a .bb directory on disk and store settings and ELT bits there.

dglazkov commented 2 months ago

For now:

dglazkov commented 2 months ago

wait... I can just reuse saved runs

dglazkov commented 2 months ago

Okay, with InspectableRun.inputs(), I think we have what we need. This is still an important issue, but we can decouple it from the Easy Mode MVP.