search

gokhanercan / gen-atomic

An LLM-based code generation framework aims to support a wide range of atomic language units, from compiled code to semi-structured markups.
3 stars 0 forks source link

Eval by Precision/Recall/F1 #16

Open gokhanercan opened 1 month ago

gokhanercan commented 1 month ago

Metrics added to ExperimentHost.py, related commit

gokhanercan commented 1 month ago

I disabled news metrics to have the pipeline green again. I was getting divide-by-zero error on integration tests. We also need to add new metrics to default reporting.

https://github.com/gokhanercan/gen-atomic/actions/runs/9150598274/job/25155562822 precision: float = (float(ccPassed) / float(ccPassed + (icCount - icPassed))) * 100 E ZeroDivisionError: float division by zero