process 100 koji builds with logjuicer and evaluate them as a data set

fedora-copr / log-detective-website

Data collection page for Log Detective AI

11 stars 7 forks source link

process 100 koji builds with logjuicer and evaluate them as a data set #106

Open TomasTomecek opened 4 months ago

TomasTomecek commented 4 months ago

Just discussed this with @jpodivin

Let's write a simple script that will:

iterate over failed production builds in koji
for every failed build, it will find last successful one in the same arch/package/chroot setup
run logjuicer on these 2 build logs
save the output
[bonus points] send the output to chatgpt/bard and see what they say about it

FrostyX commented 4 months ago

iterate over failed production builds in koji

for every failed build, it will find last successful one in the same arch/package/chroot setup

We have a script that does this in Copr. I was supposed to polish it, so it can be run via Cron but it is still to be done. If Copr would be good enough, somebody can work on the remaining steps and we can glue it together afterward.

TomasTomecek commented 4 months ago

@FrostyX wow, amazing! please share it with me, I can do that for Copr

TomasTomecek commented 4 months ago

I have made solid progress on this: https://github.com/TomasTomecek/log-detective-website/tree/logjuicer-and-llm

Logjuicer is producing amazing snippets and GPT4 can give us excellent descriptions for problems such as compilation failures, pytest failures, "autotools"-like checks