[Temporary Patch] ignore `empty` do not pass empty kv cache to runtime

neuralmagic / deepsparse

Sparsity-aware deep learning inference runtime for CPUs

https://neuralmagic.com/deepsparse/

Other

3.01k stars 176 forks source link

[Temporary Patch] ignore `empty` do not pass empty kv cache to runtime #1263

Closed bfineran closed 1 year ago

bfineran commented 1 year ago

This is a temporary change to be reverted - right now there are reported segfaults when running internal kv cache with empty placeholder arrays for ONNX graph input

test_plan: @tlrmchlsmth to verify

revert_plan: @tlrmchlsmth to confirm when revert is safe