Xilinx / inference-server

https://xilinx.github.io/inference-server/
Apache License 2.0
43 stars 13 forks source link

Batch processing added to Migraphx worker #52

Closed bpickrel closed 2 years ago

bpickrel commented 2 years ago

A major change to the Migraphx worker that adds post-version 1 features.

Coding note: the new batch processing required a significant loop refactoring in Migraphx::doRun. Pseudocode for the handling order now looks like this:

For each batch
    Call migraphx::eval(), response is array of arrays
    For each request in batch
        For each image in request
            Add result array to response
        Send response

There are two nested try-catch blocks, since some exceptions apply to the entire batch and some apply only to a single request.

gbuildx commented 2 years ago

Can one of the admins verify this patch?