search

triton-inference-server / fil_backend

FIL backend for the Triton Inference Server
Apache License 2.0
68 stars 35 forks source link

Eliminate server crash with FP64 input #119

Closed wphicks closed 3 years ago

wphicks commented 3 years ago

Respond with error to non-FP32 input and update README to note that FP32 is the only supported I/O type for the FIL backend at this time.

While attempting to fix the underlying issue here, a double free was revealed that occurred if model execution errored out at a particular point in batch processing. Because the existing implementation was somewhat complicated, it was more difficult to reason about the cause of this double free and ultimately eliminate it. In order to facilitate debugging, the TRITONBACKEND_ModelInstanceExecute method was restructured to be more readable and less error-prone. As a result, these changes are more far-reaching than strictly necessary to eliminate the crash but provide improved maintainability and stability.

Close #108. Partially address #34.

wphicks commented 3 years ago

@lowener Don't feel like you have to review this entirely on your own, but I'd love to get your thoughts on this one.