search

BuilderIO / micro-agent

An AI agent that writes (actually useful) code for you
https://www.builder.io/blog/micro-agent
MIT License
2.61k stars 206 forks source link

Integration test with mock LLM #20

Closed tpaulshippy closed 2 months ago

tpaulshippy commented 2 months ago

Purpose

Automated testing the CLI (#2)

Approach

Tried a number of approaches, but landed on these:

  1. A mechanism for recording the outputs of the LLM in a fixture for mocking.
  2. A set of tests (interactive.test.ts) that start the interactive mode and ensure that it prompts the user. Tried to go further with https://github.com/microsoft/node-pty but ran into some snags.
  3. A full end-to-end test of the non-interactive CLI execution option, which internally calls a test command so that was tricky. This one also generates the implementation so it uses the LLM mock fixture.

Open to feedback/suggestions.

steve8708 commented 2 months ago

🙌