search

Xilinx / inference-server

https://xilinx.github.io/inference-server/
Apache License 2.0
43 stars 13 forks source link

Api client #60

Closed bpickrel closed 2 years ago

bpickrel commented 2 years ago

Summary of Changes

New Migraphx test client migraphx_client.cpp that interfaces with the server via the C++ API rather than the REST interface. No tracking issue exists for this feature at time of writing.

Motivation

This client is not just an example but is intended to verify performance. What is the best turnaround time as seen by client, without the overhead of REST interface? It should be close to the optimum for the MIGraphX API.

The server's batcher is currently undergoing changes to support multi-request batches more efficiently. Specifically, this client submits a series of 1-image requests rather than one large batch-sized data array. When the updates are committed, this client should be updated to see if throughput rate improves.

gbuildx commented 2 years ago

Can one of the admins verify this patch?