[DeepSparse Evaluator API][UI] `EvaluationRegistry.resolve` implementation

neuralmagic / deepsparse

Sparsity-aware deep learning inference runtime for CPUs

https://neuralmagic.com/deepsparse/

Other

2.97k stars 171 forks source link

Closed dbogunowicz closed 8 months ago

dbogunowicz commented 9 months ago

Introduces the resolve class method for EvaluationRegistry. Additional changes:

evaluate method is now responsible for creating the model: Union[Pipeline, Any] from target:str

updated the tests
ran successfully all the tests that require lm-eval as a dependency
ran example API and CLI commands from internal docs ([Documentation] Deepsparse Evaluation API)

bfineran commented 9 months ago

see failures - looks like bad name in tests, potentially bad rebase