search

block-open-source / goose

Goose is a developer agent that operates from your command line to help you do the boring stuff.
https://block-open-source.github.io/goose/
Apache License 2.0
73 stars 12 forks source link

Hallucination/ code mangeling #60

Open Kvadratni opened 2 weeks ago

Kvadratni commented 2 weeks ago

Sometimes goose experiences a hallucination as most LLM's however due to it's agent nature it can destroy code it was working on.

image

Several fixes for this may be:

  1. Run the returned code agains a compiler before making the change
  2. Run a validator agent loop
michaelneale commented 2 weeks ago

@baxen this was a bit like I remember you discussing, the benefit of a fast local look post goose editing a file - if it is syntactically nonsense in some fast detectable way, can give that maybe as fast feedback to goose to try again.

One thing to try is tree sitter:

https://github.com/grantjenks/py-tree-sitter-languages

and just see how it parses quickly per edit