Open dglazkov opened 9 months ago
I did look into this a while ago, I wanted to see if I could intercept the Harness and dynamically inject the kits.... But I couldn't reliably work out what to replace because I would only have obfuscated code.
@paullewis any thoughts on how to provide hooks into breadboard-web
that I could add runtime injected kits?
One thought is to have a standard url/import that I could intercept in the web-server to provide all the kits.
Another could be to have a query string parameter as a list of kits that a board needs...
I thought the Breadboard Web harness automatically included all kits by default. Is that not carrying through? FWIW I think there's something we need to ponder there, because the kits aren't expressed in the JSON anywhere, so it's assumed any harness would know what - say - invoke
means. It is possible someone could have different kits which offer nodes of the same name but with different behaviours.
Yes it's possible to have two kits with the same names. There's been a long-standing namespacing question iirc.
The Web UI via the harness includes the kits in this repo, but it also ties the debugger and kits to the same release. That's not to mention, I've a heap of kits that I just can't use in the debugger.
Just bumping this back up. I'd love to be able to get kits loaded into the breadboard-web.
Some thoughts about how to do this:
fetch
to get a list of kits it needs to import.preview.ts
and config.ts
have a list of kits, I think I can have the CLI rollup
any additional kits that are specified on the CLI and then make that file available to be served from the http-server the CLI has, and then add to that kits
list.
I can do the work @paullewis for this, I just wanted to check if you have other ideas.
That seems sensible tbh. I think the BGL specifies the kits, though, so I wonder if we can load them dynamically in some way? Not spent that much time with the kits side of things.
Hmm, thinking about it some more, is the concern that we have to manually specify the kits on the CLI and then we might not have the right ones to power the board that's loaded?
The concern is that any kit that is made (light-weight, or heavy) needs to be hardcoded into the UI right now. At least as a first step - enabling you to add new kits (like my ClaudeKit) would be beneficial because we don't have to include every new little board. We can keep the current hard-coded list so it's not like you have to specify everything all the time.
re: BGL has a kits property, but I thought it was unused now - that being said, I think there was some discussion yonks ago that it's a risk to load kits dynamically without intent and there is a strong chance you won't know that a board is included in the BGL.
What if we rearrange the deps to do this:
run
API (currently invoked from inside of the bb-main
) and supply the actual board and the kitsbb-main
to use RunConfig
as a property (will include URL of the url and the kits)breadboard debug
to instantiate a bb-main
with a RunConfig
instance that is created from the command line.Ok - that's a bit different from what I have here: https://github.com/breadboard-ai/breadboard/pull/848
I will look at this suggestion though, although we have to teach bb-main
and RunConfig on how to dynmically include the kit (that's why I use rollup when I start up the CLI)
The linked PR the UI has two new HTTP endpoints:
I think #848 is fine as a workaround (and a first step), but I'd like to strive toward a more permanent, better layered solution.
totally.
@dglazkov Chatting with Metro Paul today and we had a question that we're trying to work out if we should resolve now or not.
We hard code the kits in to the Web UI, the PR mentioned here can solve that, but we're trying to work out if we should make the switch now and ensure that the UI doesn't hard-code any kits and rather we work out a way to pass them in on the CLI (or hosting) without having to do --kit
for everything.
It does raise challenges though:
1) you have to ensure for any starter project you have the kits in a local/accessible node_modules.
2) We will also hit an issue that I hit the other day - because each Kit references breadboard directly and if a Kit is not in-sync with the current version of breadboard we can't add those nodes to the graph (it errors with #scope
error). I know @aomarks was looking at a cleaner separation of Breadboard and Kit etc, I really do think we should look at this soon rather than later.
t the UI doesn't hard-code any kits and rather we work out a way to pass them in on the CLI (or hosting) without having to do --kit for everything.
Yeah, that seems directionally right!
- you have to ensure for any starter project you have the kits in a local/accessible node_modules.
Yup, this seems right, too
- We will also hit an issue that I hit the other day - because each Kit references breadboard directly and if a Kit is not in-sync with the current version of breadboard we can't add those nodes to the graph (it errors with
#scope
error).
This one is important. It comes down to coming up with a stable API for a kit, so that it can be consumed regardless of what version of Breadboard it was built with. Agreed on SRTL. Need to figure this out.
I just realized what I think we're looking for:
Shift breadboard debug
from being a "just calls into the goop that is breadboard-web
" to "uses Breadboard run
API directly and relies on breadboard-ui
to render results".
I kind of want to have breadboard run
to use the harness work you did in main breadboard, too.
okay, crazy idea time. What if breadboard debug
brings up a node proxy server that proxies all the kits from the localhost? This way, we wouldn't need to have them be all rolled up with vite.
Oh - interesting. Very interesting. Given we have breadboard proxy
working (and it's running live on one of my apps) it shouldn't be a great stretch to enable this... We know tunnelling works for keys too, so they can still be defined in the UI.
We'd have to teach the UI to also proxy the function names defined by those nodes (I'm still worried about function name clashes in that proxy).
One thing that I would have to work out is more around expectation. Would people expect it to all just be working in the browser, if it's loaded through the web ui?
I thought about it some more and we'd still need Kits bundled in with the frontend, because they have the describers for the nodes. If so, then maybe proxying is not really giving us that much.
Let's figure out how to enable adding kits at runtime for
breadboard debug
command. We can work around it for now with just adding kits tobreadboard-web
, but having this work becomes important when we have more light kits, since it enables us to supply still-WIP kits into thebreadboard debug
to actually work with them.