Support pre-built binary on runtime

[chunseoklee]

Background

We are trying to run pre-built binary on runtime. Here, you can think of pre-built binary as a kind of binary, which has the following characteristic :

• It requires specific hardware(e.g. DSP/NPU etc.) to run on target. Thus, it is usually compiled by dedicated compiler.

We would get acceleration opportunity by (heterogeneous) executing with the backend of the pre-built binary

TODOs for runtime

• [ ] Define pre-built binary format on nnpackage(#7395)
• [ ] Define operation for PBB(pre-built binary) #7501
• [ ] Loader for PBB
  • PBB model structure : Innputs -> single PBB op -> Outputs
  • Q) how to deal with multi model loading / execution ?
• [ ] Dummy backend for PBB op
• [ ] Depict acceleration scenario when the backend for pre-built binary has multiple cores / devices running in parallel
• [ ] Depict heterogenous(Mixed) execution (e.g. cpu + npu )

[chunseoklee]

Here is a picture which depicts the whole flow during model conversion from pb/tflite to nnpkg with pre-built(e.g. NPU) binary :

Here is TODOs on compiler side, IMHO.

• Define nnpkg format for partitioned models with pre-built binaries(This is both for runtime and compiler)
  • Consider the nnpkg containing both circle model and pre-built binary
• Toolchain(in one-cmd) for generating the multi-model nnpkg
  • This tool will compile pre-built binary, generate pkg information(e.g. meta info about prebuilt bin), and make an nnpkg.

[chunseoklee]

npu-engine is open-sourced : https://review.tizen.org/gerrit/gitweb?p=platform/adaptation/npu/trix-engine.git

This might facilitate an easy backend implementation.