LightEval is a lightweight LLM evaluation suite that Hugging Face has been using internally with the recently released LLM data processing library datatrove and LLM training library nanotron.
MIT License
462
stars
53
forks
source link
Updated tgi_model and added parameters for endpoint_model #208
1] Updated tgi_model to inherit from LightevalModel, including implementing all the abstract properties. Since the TGIClient and the InferenceClient produce the same format (since they both use TGI), I had TGIClient inherit from InferenceClient, and then just override the functions that use the client. In order to do that, I had to update the private functions in InferenceClient to use one underscore so the names aren't obfuscated.
2] Added two optional fields to the endpoint_model config - specifying the image_url and allowing for any other environment variables. This will allow users to update to later versions of the TGI container without having to rebuild, or to specify other properties like max_input_tokens.
1] Updated tgi_model to inherit from LightevalModel, including implementing all the abstract properties. Since the TGIClient and the InferenceClient produce the same format (since they both use TGI), I had TGIClient inherit from InferenceClient, and then just override the functions that use the client. In order to do that, I had to update the private functions in InferenceClient to use one underscore so the names aren't obfuscated.
2] Added two optional fields to the endpoint_model config - specifying the image_url and allowing for any other environment variables. This will allow users to update to later versions of the TGI container without having to rebuild, or to specify other properties like max_input_tokens.