Separation of loca LLM batch inference and Gemini evaluation from eval step

deep-diver / llamaduo

This project showcases an LLMOps pipeline that fine-tunes a small-size LLM model to prepare for the outage of the service LLM.

https://huggingface.co/papers/2408.13467

Apache License 2.0

286 stars 29 forks source link

Separation of loca LLM batch inference and Gemini evaluation from eval step #9

Closed deep-diver closed 6 months ago

deep-diver commented 6 months ago

for visibility cc'ing: @sayakpaul

currently, the eval step consists of

local LLM's batch inference
Gemini evaluation

the two sub-parts should be separated. Also, in order for Gemini evaluation to evaluate on the batch inferenced outputs, the batch inferenced outputs should be stored in a Hugging Face Dataset repository.