[luci-interpreter] Support Hybrid S4 FullyConnected kernel - Githubissues

Samsung / ONE

On-device Neural Engine

Other

429 stars 157 forks source link

[luci-interpreter] Support Hybrid S4 FullyConnected kernel #12767

Closed jinevening closed 6 months ago

jinevening commented 6 months ago

What

Let's support hybrid S4 FC kernel (input fp32, weights s4, bias fp32, output fp32).

Why

We can use circle-interpreter for end-to-end test of INT4 models. For that, we need to introduce a kernel for hybrid S4 FC in luci-interpreter.

Status of INT4 support

luci: https://github.com/Samsung/ONE/issues/12717

Examples on how to add an operator to luci-interpreter

HardSwish: https://github.com/Samsung/ONE/pull/11195
BatchMatMul: #8335 (it would be helpful to look at BMM to know how scratchpad memory is handled)

NOTE Hybrid S4 FC kernel exists in the recent TFLite (>= 2.13.1).

ljwoo94 commented 6 months ago

Since #12801 is merged, this issue can be closed