search

mllite / ml2cpp

Machine Learning Models Deployment using C++ Code Generation
BSD 3-Clause "New" or "Revised" License
8 stars 1 forks source link

Experiments with Small Devices #25

Open antoinecarme opened 3 years ago

antoinecarme commented 3 years ago

ml2cpp, as it is now, allows converting any ML model to a C++ code for inference purposes.

This code is however not yet optimized to run on small devices : these devices may have low CPU speed, low memory, low-power, no FPU, no real OS. ml2cpp Jupyter notebooks were run and inference was tested on a sparc64 or x86-64 server with gigabytes of memory and running full-featured g++ compiler.

We need:

  1. get some hardware/micro-controller : STM32, ESP32 and K210 (riscv-64) at least.
  2. Check that the generated code can be compiled with the respective gcc versions (C++-17 support, exceptions, RTTI are not always available).
  3. check the capacities of these devices (CPU + memory)
  4. Check floating point issues : is there an FPU? half-precision floats (float16), bfloat16, ...
  5. get an emulator for each device (qemu-???). use it for automated tests.

Priority 1 : Getting at least one of these devices running all these steps (feasibility). choose the one with less constraints (K210 can run a small linux)

The goal is to be able to run the model on the bare metal. No arduino. No MicroPython.

Using STM32, ESP32 or K210 is here only for tuning purposes. Their only added value is to provide a test envoironment so that C++ code will be incrementally adapted in a real-world case.

antoinecarme commented 3 years ago

STM32 + ESP32 + K210 (from left to right)

image

antoinecarme commented 3 years ago

https://jaycarlson.net/embedded-linux/

image

antoinecarme commented 2 years ago

SiFive, a pioneer of the RISC-V free and open source instruction set architecture (ISA), has announced the fruit of its work with ArchiTek Corporation: AiOnIc, an edge AI processor that combines SiFive's E3-series RISC-V cores with ArchiTek's Intelligence Pixel Engine (aIPE).

To be tested (when available) :

https://www.hackster.io/news/sifive-architek-announce-high-efficiency-sub-1w-edge-ai-processor-the-aionic-1d8aa537f77f

image

antoinecarme commented 2 years ago

Kendryte K510 Edge AI Chip as a Triple-Core RISC-V Part with 3 TOPS NPU. Has FP16 support

To be tested (when available) :

https://canaan.io/product/kendryte-k510

image

antoinecarme commented 2 years ago

Alibaba On The Bleeding Edge Of RISC-V With XT910. Has FP16 support.

To be tested (when available) :

https://www.nextplatform.com/2020/08/21/alibaba-on-the-bleeding-edge-of-risc-v-with-xt910/

image

image

antoinecarme commented 1 year ago

https://www.mdpi.com/1999-5903/14/12/363

image

image

antoinecarme commented 1 year ago

https://stackoverflow.com/questions/5710942/c-on-small-footprint-microcontrollers

https://www.open-std.org/jtc1/sc22/wg21/docs/TR18015.pdf