hotg-ai / rune

Rune provides containers to encapsulate and deploy edgeML pipelines and applications
Apache License 2.0
134 stars 15 forks source link

Typecheck known model types against their inputs and outputs #330

Open Mohit0928 opened 2 years ago

Mohit0928 commented 2 years ago

I'm typing the wrong input/output type and dimensions in the Runefile.yml, but even then, it's building a rune. @Michael-F-Bryan, It should throw an error if I write a wrong input/output type and dimension in the Runefile.yml. Below is an example:

Model info:

Inputs:
        serving_default_input:0: f32[1,192,192,3]
Outputs:
        StatefulPartitionedCall:0: f32[1,1,17,3]

Typing a wrong Runefile.yml:

version: 1
image: runicos/base

pipeline:
  image:
    capability: IMAGE
    outputs:
      - type: u8
        dimensions: [1,192,192,3]
    args:
      height: 192
      width: 192
      pixel-format: "@PixelFormat::RGB"
  normalizer:
    proc-block: "hotg-ai/rune#proc-blocks/image-normalization"
    inputs:
      - image
    outputs:
      - type: f32
        dimensions: [1,191,191,3]  #wrong output dimensions, it should be [1,192,192,3]
  pose:
    model: "./movenet_singlepose_lightning_3.tflite"
    inputs:
      - normalizer
    outputs:
      - type: u32             #wrong output type, it should be f32
        dimensions: [1,17,17,3]
  serial:
    out: SERIAL
    inputs:
      - pose
Michael-F-Bryan commented 2 years ago

This is something I'd like to do as well.

I've been reluctant to until now because it means the compiler frontend crate will now add TensorFlow Lite (either tflite or librunecoral) as a dependency... But that's a bloody big dependency and really hurts compile times if you want to just use the compiler frontend (e.g. Forge).

To implement this, during the "lowering" phase we need to associate models with minetypes based on the filename's extension. Then during the "typecheck" phase we need to add a pass which looks for TensorFlow Lite models, loads them, and checks input/output types.