coderustic
commented
1 month ago Looking at this latest evaluation of unit test generation, it can be easier to build different combination of strategies between -
1) Prompt design a) To have NL comprehension vs. Code snippets b) To include code features and which features provides better results? 2) Incontext learning - Choosing between CoT vs. RAG
While cover-agent continues to be a command line tool, splitting the CLI interface from a API interface could help expose the Pipeline aspects to CLI as well as it also enables anyone to build their own strategy.
When you look at the CoverAgent + UnitTestGenerator, there is a sequence of steps composed into a Pipeline. We should expose an ability to build the pipeline by composing the stages of the pipeline and run via CoverAgent.
Having an API enables fast experimentation for building different strategies for eg: should we include natural language comprehension of the code vs the actual code snippet.
Having an API enables programmatic approach for users to build their own custom agent rather than dealing with a complex combination of configuration options.
To keep up with the research and enable fast experimentation of different strategies, so cover-agent could become a framework for any one to do fast experiments to decide which pipeline makes sense for a particular codebase and then deploy the CI job instead of dealing with config options.
Abstracting into different interfaces means we can compose different strategies and build a pipeline 1) Should I include natural language comprehension? 2) Should I include code features? Focal method, focal class features - constructors, other methods, class definitions of arguments of focal method etc 3) Is there a tool that can help easily figure out dependencies of focal method and extract code features of dependencies instead of user choosing them via --include-additional–files
While this issue is only for enabling Pipeline creation API, enabling NL comprehension or adding Code features could be different issues.