CMSIS-NN Feature: Dependency resolution issue

[PhilippvK]

There is a flaw iin the current implementation of the cmsisnn issue:

• This feature requires the config (or chache key) cmsisnn.lib which is then passed to the tflm.optimized_kernel_libs configuration variable.
• While this works out when only using CMSIS-NN on a single target (e.g. ARM corstone300 FVP) we get into trouble if we also want to use other architectures, as each of them is building a specific static library for CMSISNN.

• Do achieve this in mlonmcu setup builds multiple static libraries leading to the following dependency cache (deps/cache.ini):

  [x86]
  cmsisnn.lib = /tmp/mlonmcu_env_test/deps/install/cmsisnn_x86/libcmsis-nn.a
  
  [dbg,x86]
  cmsisnn.lib = /tmp/mlonmcu_env_test/deps/install/cmsisnn_x86_dbg/libcmsis-nn.a
  
  [riscv]
  cmsisnn.lib = /tmp/mlonmcu_env_test/deps/install/cmsisnn_riscv/libcmsis-nn.a
  
  [dbg,riscv]
  cmsisnn.lib = /tmp/mlonmcu_env_test/deps/install/cmsisnn_riscv_dbg/libcmsis-nn.a
  
  [arm]
  cmsisnn.lib = /tmp/mlonmcu_env_test/deps/install/cmsisnn_arm/libcmsis-nn.a
  
  [arm,dbg]
  cmsisnn.lib = /tmp/mlonmcu_env_test/deps/install/cmsisnn_arm_dbg/libcmsis-nn.a

• The required flags for the architecture get be get from the via target.get_arch() method
• However as all features are initialized in the beginning before any targets exists, the cache variable cmsisnn.lib can not be resolved because it is missing the flags x86/arm/riscv and dbg from the compile stage... -> The actual issue

A workaround for this issue is explicitly passing the value of cmsisnn.lib via the command line: --config cmsisnn.lib=/tmp/mlonmcu_env/deps/install/cmsisnn_x86_dbg/libmuriscv-nn.a

To tacke the issue we have to resolve one/both of these problems:

• A. Currently Features are "globally" applied to a session, e.g. every run receives the same set of features (Features are instaiated before runs are created), this would not allow mixing different architectures for a single feature. Instead we would move the feature initialization the every single run, which has the caveat that we end up with a totals of #runs x #features feature objects.
• B. Circular Feature dependencies: We can not just stick to the following initialization order: Feature -> Frontend -> Backend -> Framework -> Target -> MLIF -> Postprocess as some features depend on the Target. However as Frontend/Backend/Framework features exists, we can not just move the Feature initialization to a later stage.

Another option which might be feasible is the following:

Instead of the actual value of cmsisnn.lib config/cache variable we just pass some sort of reference to tflm.optimized_kernel_libs which is then resolved when tflm.optimized_kernel_libs is accessed, e.g. in TfLiteFramework.get_cmake_args(). We should keep this in mind when tackling #15 as this might help to get rid of A. and B..

---

The issue would also apply to muriscvnn, as we it would be great if we could use the scalar version on other platforms as well for comparisons.

[PhilippvK]

An alternative approach would be building the libcmsis-nn.a (or libmuriscv-nn.a) on demand when compiling the target software with cmake. However I would prefer to have those libs prebuild to only link against them.

[PhilippvK]

I would like to build cmsis-nn, muriscv-nn,... together with the target software instead of pre-building them during mlonmcu setup as we do not have another choice for non-MLIF platforms (e.g. espidf and further platforms which might get added in the future). As those libs are quite small the increate of compile time should be negligible.

However to use those features in combination with the tflite-micro-compiler they needs to be a version of tflmc.exe built with e.g. muriscvnn support -> As long as we only execute the pre-interpreter on the host (x86) this should me possible as only one version of muriscvnn.lib would exists.

As mentioned in #17 a long term goal would be using TFLMC in the target architecture to get more accurate arena estimations and enable target-specific features (e.g. vext). This would not work out nice with the mentioned workaround...

[PhilippvK]

Closing because not meaningful