Closed wongjingping closed 2 months ago
Enable testing of inference using TGI via the api runner. We only need to change the way the request parameters are formatted, and facilitate this simple switching via the new flag --api_type Updated README to show how to use it.
--api_type
Enable testing of inference using TGI via the api runner. We only need to change the way the request parameters are formatted, and facilitate this simple switching via the new flag
--api_typeUpdated README to show how to use it.