Better LLM Context for packages and imports

stackblitz / bolt.new

Prompt, run, edit, and deploy full-stack web applications

https://bolt.new

MIT License

5.21k stars 1.05k forks source link

Better LLM Context for packages and imports #28

Open CodeFromAnywhere opened 2 weeks ago

CodeFromAnywhere commented 2 weeks ago

Is your feature request related to a problem? Please describe:

Congrats on the launch! Very cool stuff, but one immediate limitation I noticed is you don't have realtime info about packages. like what's inside them...

Describe the solution you'd like:

I'd like the llm to use a tool so it can reason over realtime contents of the package

Describe alternatives you've considered:

The alternative is to just use large packages that it knows, but that's quite a limitation!

Additional context:

I'm building http://swcapi.com and http://npmjz.com let me know if you need help with that!

Nemikolh commented 1 week ago

Hey @CodeFromAnywhere ! :wave:

Do you mean to show something like which npm packages are installed in the project? If yes, this is visible in the package.json.

CodeFromAnywhere commented 1 week ago

no, that's not what I mean.

I mean that your ai hallucinates on the contents of packages. it does not use any context of the actual stuff inside of packages. for bigger packages it's totally fine, but what about smaller ones that aren't represented well in your dataset of the model, or packages owned by the user? you need this additional context to have an ai chatbot that does not hallucinate.

im creating an api for having this context

Nemikolh commented 1 week ago

I think there's a misunderstanding. Packages are fetched from the npm registry the same way they're fetched on https://stackblitz.com. They are not generated by bolt.

CodeFromAnywhere commented 1 week ago

I guess the things I say are a bit ambiguous... 😅

The halicunation occurs in the code generation part.

Of course I don't doubt the content of the node_modules folder is sound.

What I mean is that the LLM often hallucinates which functions it should use from packages it installs. Because of that, it generates code that breaks.

Have you thought about this?

As an example, see this prompt with the result:

It hallucinates ^1.0.0 which doesn't exist.

It hallucinates filterArray which doesn't exist. It didn't even check! I also don't see a red line, probably because package installation failed.

Edit: seems it gets stuck here for over 2 minutes now which may be why no red line for edge-util.

Nemikolh commented 1 week ago

Oh thank you so much for the clarification! So sorry, I totally understand what you mean.

I think we could definitely improve on that area to help the AI model so that it generates only known versions. For imports it's a bit more tricky but there's a plenty of ideas we want to experiment.

I'll add this to our internal tracker and we'll keep you posted.