[AQUA][Evaluate] Added the EvaluationServiceConfig class to manage evaluation service configurations. - Githubissues

oracle / accelerated-data-science

ADS is the Oracle Data Science Cloud Service's python SDK supporting, model ops (train/eval/deploy), along with running workloads on Jobs and Pipeline resources.

https://accelerated-data-science.readthedocs.io/

Universal Permissive License v1.0

87 stars 43 forks source link

[AQUA][Evaluate] Added the EvaluationServiceConfig class to manage evaluation service configurations. #940

Closed mrDzurb closed 1 month ago

mrDzurb commented 1 month ago

Description

Added the EvaluationServiceConfig class along with supporting structures to manage evaluation service configurations. This includes methods to merge evaluation inference parameters, retrieve model parameters based on framework and version, and filter shapes based on provided criteria. The implementation ensures flexibility and scalability for various frameworks and tasks.

Dependency

This PR has a dependency on the Introduce Global Configuration for LLM Evaluation

Tests

github-actions[bot] commented 1 month ago

⚠️ This PR changed pyproject.toml file. ⚠️

PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in pyproject.toml.
PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in pyproject.toml.

github-actions[bot] commented 1 month ago

⚠️ This PR changed pyproject.toml file. ⚠️

PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in pyproject.toml.
PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in pyproject.toml.

github-actions[bot] commented 1 month ago

⚠️ This PR changed pyproject.toml file. ⚠️

PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in pyproject.toml.
PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in pyproject.toml.

github-actions[bot] commented 1 month ago

⚠️ This PR changed pyproject.toml file. ⚠️

PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in pyproject.toml.
PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in pyproject.toml.

github-actions[bot] commented 1 month ago

⚠️ This PR changed pyproject.toml file. ⚠️

PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in pyproject.toml.
PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in pyproject.toml.