search

OpenBMB / ChatDev

Create Customized Software using Natural Language Idea (through LLM-powered Multi-Agent Collaboration)
https://arxiv.org/abs/2307.07924
Apache License 2.0
25.42k stars 3.19k forks source link

Proposal: adding Unit tests #143

Closed thomasparsons closed 12 months ago

thomasparsons commented 1 year ago

Hi ChatDev! enjoying the tool so far, though I've found that it's struggled to make a working version of a slightly more complex problem, and one part of that is the code not even running without code changes!

I think if we could add unit testing to this, it could reduce this problem, and I've had great success with AI-generated tests. I have asked this via the prompts, but it's not actually happening, so perhaps this needs to be in the code instead?

I Propose it is the engineer's responsibility to create test files for each file it creates (i.e. main.test.py), and the responsibility of the QA? to run the test - though one might argue it should be run before the code is submitted...

Alphamasterliu commented 1 year ago

Hello! Thank you very much for this substantial suggestion, and we will certainly consider it. Unfortunately, due to recent scheduling constraints with our development team, we currently don't have immediate plans for unit testing. However, for optimizing before software generation, you might consider using the "--config 'Human'" option to enable aHuman-Agent-Interaction mode. Please stay tuned for our upcoming features, and we believe they will be arriving in the near future.😊😊😊