Pipeline/eval

[deep-diver]

this PR basically runs the programs written in a jupyter notebook for model evaluation #5

when it is run, the following is printed out

eval on 0...similarity_score: 12, precision_score: 20
eval on 1...similarity_score: 30, precision_score: 50
eval on 2...similarity_score: 85, precision_score: 90
eval on 3...similarity_score: 50, precision_score: 25
eval on 4...similarity_score: 70, precision_score: 80
eval on 5...similarity_score: 85, precision_score: 95
eval on 6...similarity_score: 90, precision_score: 95
eval on 7...similarity_score: 10, precision_score: 0
eval on 8...similarity_score: 25, precision_score: 50
eval on 9...similarity_score: 30, precision_score: 10
eval on 10...similarity_score: 30, precision_score: 50
eval on 11...similarity_score: 45, precision_score: 20
eval on 12...similarity_score: 20, precision_score: 0
eval on 13...similarity_score: 80, precision_score: 85
eval on 14...similarity_score: 60, precision_score: 85
eval on 15...similarity_score: 10, precision_score: 0
qualification: False, avg_similarity_scores: 45.75, avg_precision_scores: 47.1875

and it returns the following values for the downstream jobs

qualification: False, avg_similarity_scores: 45.75, avg_precision_scores: 47.1875

This code runs the text generation inference with batch size of 1. In the later follow-up PR, text generation with arbitrary batch sizes should be supported. To do this, we need the following code modifications:

• batch generation on a given test dataset with local language model (Gemma)
• simultaneous API call to service language model (Gemini) for the evaluation on each generated output

[deep-diver]

@sayakpaul

addressed comments : local LLM (gemma)'s batch inference with dataset.map() approach. : makes simultaneous gemini API calls on the batch generated outputs

[deep-diver]

@sayakpaul

addressed your comments!

[sayakpaul]

Thanks for iterating!