search

possee-org / genai-numpy

MIT License
4 stars 6 forks source link

ENH: Log Processing for example generation #90

Closed bmwoodruff closed 1 week ago

bmwoodruff commented 1 week ago

This adds some functionality to extract new examples from the logs. Currently only 590/829 logs can safely have examples extracted. I'll be working to get this up to hopefully well above 90%. Coupled with the reviewtools.py script, this should enable us to take Nebari output and inject it directly into the code. I needed the massive number of logs from Llama3 so that I have enough information to realize what common pitfalls to look out for.

This is part of the work needed for #77.

bmwoodruff commented 1 week ago

It's still a work in progress, so I'm going to leave it open as a PR for a day or two while I continue to make commits.