ChufanSuki
commented
1 month ago model conversion: transform a source code into the semantic equivalent in another framework format and generate a piece of model construction code.
It's feasible because all DL frameworks have tensor-oriented computation graphs. However, the following non-trivial challenges exist:
- discrepancies in both the computation graph constructs and the supported features between DL frameworks. E.g. PyTorch provides $log1p$, not all DL frameworks support both $NCHW$ and $NHWC$ tensor layouts.
- computational structure saved in a model file can be different and complex. E.g. a PyoTrch model file stores a dynamic computation graph "defined-by-run". Therefore, reverse syntax transformation is important for a source model to recover the original DL constructs.
- There are a large number of popular DL frameworks, bringing potential significant engineering efforts. Such a problem is very similar to that in compiler construction to support various front-end programming languages and back-end architectures. We could refer to compilers on the design and implementation.
MMdnn's approach:
- IR-based methodology. The source model is first transformed into an intermediate computation graph represented by our IR and then to the target format, using a node-to-node translation technique similar to the compiler’s instruction selection.
- Reverse syntax transformation on the source model for better semantic comprehension via a pattern matching similar method
- Extensible conversion architecture from the compilation point of view. It decouples the whole conversion pipeline into three major phases: source model reconstruction, intermediate model transformation, and target model generation
https://www.microsoft.com/en-us/research/uploads/prod/2019/11/mmdnn.pdf
https://github.com/microsoft/MMdnn/tree/master