search

tensorflow / tflite-micro

Infrastructure to enable deployment of ML models to low-power resource-constrained embedded targets (including microcontrollers and digital signal processors).
Apache License 2.0
1.74k stars 770 forks source link

Dynamic range quantization support in TFLM #2543

Closed Doomski99 closed 1 month ago

Doomski99 commented 2 months ago

Hello,

I'm having trouble testing Dynamically quantized models on my DSP simulator. I figured it might be not supported on TFLM. I spent much time searching for an official statement about Dynamic Quantization support for TFLM to no avail. The PTQ page of tensorflow summarizes the different quantization options with recommended hardware. But does that also entail "supported hardware" too?

Can I get an official statement about DQ support on TFLM? Does it depend on the chip maker?

Thanks in advance.

rascani commented 2 months ago

You are correct that TFLM does not currently support dynamic range quantization. This has primarily been because the microcontrollers we have supported lacked hardware floating point acceleration, so inference performance with DRQ would be poor. We've generally recommended full integer quantization either using PTQ or QAT.

I'm leaving this open for myself to add some documentation to highlight this requirement.

github-actions[bot] commented 1 month ago

"This issue is being marked as stale due to inactivity. Remove label or comment to prevent closure in 5 days."

github-actions[bot] commented 1 month ago

"This issue is being closed because it has been marked as stale for 5 days with no further activity."