search

eficode-academy / team-collaboration-side-project

0 stars 0 forks source link

codeball.ai - AI powered code review #81

Open ncsokas opened 7 months ago

ncsokas commented 7 months ago

Codeball finds bugs in your Pull Requests, lets you ship faster and with higher confidence. The Codeball AI is trained on millions of code contributions to recognize risky code changes.

┆Card is synchronized with this Github issue by Unito

Hexamo commented 7 months ago

Makes me kind of nervous that their website is down. 😓 https://codeball.ai/

The action has not been worked on since 2022? The founders has left codeball in jan 2023 https://www.linkedin.com/in/gustavwestling/ https://www.linkedin.com/in/kirilv/ Their linkedin company has no people who want so associate with it? https://www.linkedin.com/company/sturdy-devtools/people/ Many of the last commits before the project seems to be abandoned does not even use codeball.

Im not saying that its a bad product. Maybe just that its, abandoned?

Hexamo commented 7 months ago

Originallay we planned on looking at Codeball.ai. This has been discontinues.

We will now be using CodeRabbit. This is nice as the "pro" part is free for open source projects. This seems to depend on contacting them? To start with we will use the trial pro version

This will start on a test repo. Right now we will do it on a

Uses LLM gpt 4 and 3.5

No way to export the webui settings as a yamlfile through a PR???

First test: bug submodules

This was pretty impressive, but unfortunately we ran into limits based on it being in a trial. It found some spelling mistakes in a readme.md file. but nothing more advanced.

It did come with some "nitpicks" that were outside range. Though, one of these are breaking.

secound test: Opensource PR someone has made.

Taking longer than expected. Gives some good comments on the code and readme. There were no real issues with the PR request.

In general pretty impressive. The poems are a nice touch.

Third test: Yaml file

We have added a .coderabbit.yaml file to the repo that the tool should use in relation to PRs for the project.

Fourth test: Python code

It does not seem to being able to see references to mulitple items. Trying to correct variables that are used further down. Trying to remove whole lines of code because the variable created is not used. But also removing the call. It is kind of blind to the bigger picture

Trying to chat with it. Asking if the changes that it is suggesting makes sense? There are some suggestions that on the tin looks correct, but in context makes no sense.

The more we are playing around with the tool in relation to coding the less impressive it is. If we chat with it regarding its recommendations then it often comes with worse results, and it cannot add the text suggestions to the code. If you chat with the AI and specificly ask it, it can add the changes.

Conclusion

The tools integrations in Github is good. But the coding part could be a lot better. It is nice for it to be able to make suggestions in-line. It gives some okay summaries. But for coding suggestions tools like Copilot is easier and nicer to use.