huggingface / lighteval

LightEval is a lightweight LLM evaluation suite that Hugging Face has been using internally with the recently released LLM data processing library datatrove and LLM training library nanotron.
MIT License
467 stars 54 forks source link

Added Namespace parameter for InferenceEndpoints, added option for passing model config directly #147

Closed shaltielshmid closed 2 months ago

shaltielshmid commented 2 months ago

1] I added the namespace parameter to the endpoint model config, to allow deploying endpoints from an organization that isn't your private user.

2] I added the option to specify the model config directly from python code, with a simple check to see if it was given. (Note: I unindented everything after the yaml.safe_load, since after the yaml was loaded there was no need to remain inside the scope of accessing the file).

Sending you sample code to test on slack.

shaltielshmid commented 2 months ago

Great!! I'll add in the docs later tonight or tomorrow.

shaltielshmid commented 2 months ago

Thanks for the comments! Added in the doc strings :)