search

huggingface / lighteval

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends
MIT License
689 stars 78 forks source link

add: dtype management (#117) #120

Closed bilgehanertan closed 6 months ago

bilgehanertan commented 6 months ago

Resolves #117

clefourrier commented 6 months ago

Hi! Thanks for this PR! You also need to add it to the InferenceEndpointModelConfig class, and grab it from the kwargs in the main to initialize the class :)

bilgehanertan commented 6 months ago

I am not sure what you mean about the kwargs part. Isn't it directly load model_dtype?

clefourrier commented 6 months ago

You need to make sure that the argument is passed at the different places where the model config is created - I can give you more pointers tmr if you need :)

clefourrier commented 6 months ago

Hi, A more in depth version of this PR was done in #124 (notably by supporting all possible dtypes), so we will probably merge the other one. Thanks a lot for taking a look :)