In the original version of ml_on_mcu we have managed build of the target software as follows:
Create a single build directory for each different set of features, e.g. build, build_dbg, build_muriscvnn_dbg,...
Make sure that only one program can be build at a time (no parallelism allowed)
Copy over the resulting ELF file to another place before building the next executable.
This has the following advantages:
The runtime libraries for TVM and TFLM only need to be build once per build directory
Easy to debug CMake/Make problems as build directories do not change
However there are some drawbacks:
No option to exploit parallelism when building target software
Risk of inconsistencies due to mixing up runtime libs.
In the new MLonMCU the approach is currently as follows:
Create a new build directory for every single run e.g. temp/sessions/0/runs/0/mlif_build
While this resolves all issues listed above this leads to a few problems as well:
Need to rebuild the runtime lib for the chosen backend every single time (at least 30 seconds for TFLM) which makes everything slower and leads to more disk usage.
As build directories change with every run, it is much harder to debug CMake/Make-related issues. The build process is split up into separate steps and we explicitly need to provide CMake the prebuilt libraries.
How to overcome these limitations?
My proposal:
Split up the MLIF into two parts:
1. Framework/Backend Runtime Libraries (Dependencies)2. Actual Target Software and codegen results
Build 1. only once per framework/backend combination (for specific features such as dbg, muRiscvNN etc.) and only build 2. in every invocation of the flow which should be much faster.
Will this work out?
AFAIK we will get into issues with the TVM CRT as there is a dependency cycle... (CRT needs kernels and kernels need CRT)
We also need to decide where the artifacts of 1. should be stored and when they are created: Either create them in the deps directory during mlonmcu setup for every possible combination of "flags" or create them on-demand in $MLONMCU_HOME/temp/mlif/?
PhilippvK
commented
1 year ago
In the original version of
ml_on_mcuwe have managed build of the target software as follows:build,build_dbg,build_muriscvnn_dbg,...This has the following advantages:
However there are some drawbacks:
In the new MLonMCU the approach is currently as follows:
temp/sessions/0/runs/0/mlif_buildWhile this resolves all issues listed above this leads to a few problems as well:
How to overcome these limitations?
My proposal:
dbg,muRiscvNNetc.) and only build 2. in every invocation of the flow which should be much faster.Will this work out?
depsdirectory duringmlonmcu setupfor every possible combination of "flags" or create them on-demand in$MLONMCU_HOME/temp/mlif/?