neuralmagic / deepsparse

Sparsity-aware deep learning inference runtime for CPUs
https://neuralmagic.com/deepsparse/
Other
2.97k stars 171 forks source link

[Evaluator][Feature Branch] End to end testing and API improvements #1410

Closed dbogunowicz closed 9 months ago

dbogunowicz commented 10 months ago

Feature Description

This PR introduces several mini-features that were necessary for one of three reasons: they improve the UI, enable the previous stand-alone implementation to work together as a single module, or are tests.

  1. Redesigning the CLI to match the agreed-upon design. CLI and API are now separate entities. CLI is responsible for printing and saving the results.
  2. Introducing the object Result that holds both results of the evaluation in the "raw" form, as well as in the structured form, shared across the integrations.
  3. Adding more tests, mostly integration tests for API and CLI.