search

cpetig / tflite_micro_compiler

generate tflite micro code which bypasses the interpreter (directly calls into kernels)
Apache License 2.0
76 stars 26 forks source link

Update tflite-micro-compiler to work with recent tflite-micro #69

Open panickal-xmos opened 1 year ago

panickal-xmos commented 1 year ago

Hi,

We at XMOS have been doing some work internally to move tflite-micro-compiler up to work with latest tflite-micro.

We would be happy to contribute the changes. It contains some hardware-specific changes at the moment. If a PR is of interest, we can put some effort into cleaning it up. Would that be of interest?

Best regards, Deepak

rafzi commented 1 year ago

Hello and thanks for reaching out!

There already were efforts to update this tool by @yair-ehrenwald and @andrewstevens-infineon.

I'd suggest that you send a PR including your specific changes without much effort to the master branch so that we can all take a look at your approach and can find the most suitable solution.

panickal-xmos commented 1 year ago

Thank you @rafzi! Will tidy it up a bit and put up a PR.

andrewstevens-infineon commented 1 year ago

Hi Deepak

Sounds great. Please do add PR with your goodies.

Here @ Infineon currently have a pretty extensive internal fork that is long overdue to be push-ed as PR. This adds better arena optimization, generation of static OpData (user_data) record and support for Prepare-phase selection of kernel variants (to allow link-time elimination of unused kernel variants.

Current plan is to get it out after the first productive release (part of Modus Toolbox 3.0 / MTB-ML 2.0 for our PSoC product-line) and refactor so it bases on latest+greated tflite(u) without a lot of in-house dependencies. Sounds like your PR would cover some of this.

If you're interest in smore details I've promised to do a brief walk-through at the next tflite(u) SIG.
I'd be delighted to collaborate/coordinate on further development / ideas. There's plenty more useful stuff to do...

panickal-xmos commented 1 year ago

Thank you @andrewstevens-infineon! It sounds like you have made much more progress with the fork. :) Will be great to see your PR. I have got ours working with the latest tflite-micro with few changes. Due to the addition of MicroContext recently in tflite-micro, we apply a small patch to tflite-micro, to avoid linking in all of that. Would be happy to work together on future ideas!

I was not aware of tflite-micro SIG. I'll join the next SIG micro meeting. Looking forward to the walk-through!

andrewstevens-infineon commented 1 year ago

Glancing at your changes looks like we've been covering similar ground w.r.t ScratchBuffer handling. Our version bases on tflite(u) from 2022-02-22 (same version ARM used for the -03 release of their ethos-U SW stack).

However the approach is a little different - rather than expand the "peeking" into the interpreter internals we went the route of patching the kernel to add callback hooks we can intercept at relevant entry-points. The hope being that these would probably be more stable than the internals implementing them.

panickal-xmos commented 1 year ago

I have put up the PR here. Please have a look through.

Tflite-micro has introduced MicroContext class, which is now the entry point to the Allocate methods. Linking it in would link in the allocator, micro_graph etc, thereby losing the benefit of reducing code size. To avoid this, two changes are done.

Missing functionality Some ops are not supported at the moment, as they link in unsupported methods. Haven't looked into them yet, as they are not an immediate priority for us. These are: