Codium-ai / cover-agent

CodiumAI Cover-Agent: An AI-Powered Tool for Automated Test Generation and Code Coverage Enhancement! šŸ’»šŸ¤–šŸ§ŖšŸž
https://www.codium.ai/
GNU Affero General Public License v3.0
3.96k stars 262 forks source link

CPP Language support with googletest #81

Closed amzechochar closed 3 weeks ago

amzechochar commented 3 weeks ago

Although lcov isn't supported yet, I was able to get around it by converting it to cobertura. lcov_cobertura test.info -o ./coverage.xml cover-agent correctly handles the coverage report, but it is failed in recognizing testing_framework and failed in initial test suite analysis

Version: f04528 $ git log commit f04528f785ea32c98012ba83c72950b4808592d7 (HEAD -> main, tag: 0.1.38, orig

poetry run cover-agent \

--source-file-path "templated_tests/cppapp/SimpleMathOperations.cpp" \ --test-file-path "templated_tests/cppapp/test_simple_math_operations.cpp" \ --code-coverage-report-path "templated_tests/cppapp/coverage.xml" \ --test-command "sh templated_tests/cppapp/dotest.sh" \ --coverage-type "cobertura" \ --desired-coverage 50 \ --max-iterations 4 \ --model "gpt-4o" 2024-06-02 14:44:54,072 - cover_agent.UnitTestGenerator - INFO - Running build/test command to generate coverage report: "sh templated_tests/cppapp/dotest.sh" Streaming results from LLM model...

language: c++
testing_framework: unknown
number_of_tests: 0
test_headers_indentation: 0

Streaming results from LLM model...

language: c++
testing_framework: unknown
number_of_tests: 0
relevant_line_number_to_insert_after: 0

Streaming results from LLM model...

language: c++
testing_framework: unknown
number_of_tests: 0
relevant_line_number_to_insert_after: 0

Streaming results from LLM model...

language: c++
testing_framework: unknown
number_of_tests: 0
relevant_line_number_to_insert_after: 0

2024-06-02 14:45:30,932 - cover_agent.UnitTestGenerator - ERROR - Error during initial test suite analysis: Failed to analyze the relevant line number to insert new tests Traceback (most recent call last): File "/home/user1/cover-agent/cover_agent/UnitTestGenerator.py", line 271, in initial_test_suite_analysis raise Exception("Failed to analyze the relevant line number to insert new tests") Exception: Failed to analyze the relevant line number to insert new tests

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "", line 1, in File "/home/user1/cover-agent/cover_agent/main.py", line 138, in main test_gen.initial_test_suite_analysis() File "/home/user1/cover-agent/cover_agent/UnitTestGenerator.py", line 277, in initial_test_suite_analysis raise "Error during initial test suite analysis" TypeError: exceptions must derive from BaseException

cat SimpleMathOperations.cpp

include "SimpleMathOperations.h"

include

int SimpleMathOperations::add(int a, int b) { return a + b; }

int SimpleMathOperations::subtract(int a, int b) { return a - b; }

int SimpleMathOperations::multiply(int a, int b) { return a * b; }

double SimpleMathOperations::divide(int a, int b) { if (b == 0) { throw std::invalid_argument("Division by zero is not allowed."); } return static_cast(a) / b; }

int SimpleMathOperations::fibonacci(int n) { if (n <= 1) { return n; } return fibonacci(n - 1) + fibonacci(n - 2); }

cat SimpleMathOperations.h

ifndef SIMPLEMATHOPERATIONS_H

define SIMPLEMATHOPERATIONS_H

class SimpleMathOperations { public: int add(int a, int b); int subtract(int a, int b); int multiply(int a, int b); double divide(int a, int b); int fibonacci(int n); };

cat test_simple_math_operations.cpp

include <gtest/gtest.h>

include "SimpleMathOperations.h"

// define a test suite, named as SimpleMathOperationsTest class SimpleMathOperationsTest : public ::testing::Test { protected: SimpleMathOperations mathOps; };

// below is testcase based on googletest framework // test add method TEST_F(SimpleMathOperationsTest, Add) { EXPECT_EQ(mathOps.add(10, 5), 15); EXPECT_EQ(mathOps.add(-1, 1), 0); EXPECT_EQ(mathOps.add(0, 0), 0); } // below is run all googletest framework cases // use InitGoogleTest int main(int argc, char **argv) { ::testing::InitGoogleTest(&argc, argv); return RUN_ALL_TESTS(); }

mrT23 commented 3 weeks ago

I was not able to reproduce your results.

Triple-check that the file locations are right, and that you are giving the correct command. My guess it that for some reason the model got an empty test file.

You can enable WANDB logging to log the prompts, and validate that you are indeed reading the correct test file. I am 98% you were not

image image

amzechochar commented 3 weeks ago

Thanks, can not reproduce now, it does work.

Kaviarasu961 commented 2 weeks ago

I could not find the templates for cpp? is it deprecated? @mrT23

ykondrashyn commented 2 weeks ago

@mrT23 Could you please clarify if cpp is supported?

mrT23 commented 2 weeks ago

cpp is supported.

you are welcome to compile the example above to a PR: https://github.com/Codium-ai/cover-agent/issues/81#issue-2332380679 To an example project, where people could run cover agent on cpp

EmbeddedDevops1 commented 2 weeks ago

FYI, @ykondrashyn supporting LCOV is at the top of our TO DO list.